2020-05-24T13:29:06.970 INFO:root:teuthology version: 1.0.0-4413d31 2020-05-24T13:29:06.983 DEBUG:teuthology.run:Teuthology command: teuthology --block --description rgw/multisite/{clusters.yaml frontend/civetweb.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml} --name kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi --archive /home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087414 --verbose --owner scheduled_kchai@teuthology --lock -- /tmp/teuthology-worker.ojquiu7k.tmp 2020-05-24T13:29:07.039 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-24T13:29:07.116 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087414 branch: nautilus description: rgw/multisite/{clusters.yaml frontend/civetweb.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml} email: tchaikov@gmail.com first_in_suite: false job_id: '5087414' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi nuke-on-error: true os_type: centos overrides: admin_socket: branch: nautilus ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= rgw curl low speed time: 300 rgw data log num shards: 4 rgw md log max shards: 4 rgw sync log trim interval: 0 global: osd heartbeat grace: 40 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 osd fast shutdown: false osd.0: osd_max_omap_entries_per_request: 10 osd.1: osd_max_omap_entries_per_request: 1000 osd.2: osd_max_omap_entries_per_request: 10000 log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 6fe217b2dddadf1baae6a015a57f83f4283a6078 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck wait-for-scrub: false ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: notcmalloc sha1: 6fe217b2dddadf1baae6a015a57f83f4283a6078 rgw: compression type: random frontend: civetweb rgw-multisite: realm: is default: true name: test-realm zonegroups: - endpoints: - c1.client.0 is_default: true is_master: true name: test-zonegroup zones: - endpoints: - c1.client.0 is_default: true is_master: true name: test-zone1 - endpoints: - c2.client.0 is_default: true name: test-zone2 - endpoints: - c1.client.1 name: test-zone3 - endpoints: - c2.client.1 is_pubsub: true name: test-zone4 workunit: branch: nautilus-py3 sha1: 2e1d102f0cb696365106419cf1e42700ba50a8ff owner: scheduled_kchai@teuthology priority: 90 repo: https://github.com/ceph/ceph.git roles: - - c1.mon.a - c1.mgr.x - c1.osd.0 - c1.osd.1 - c1.osd.2 - c1.client.0 - c1.client.1 - - c2.mon.a - c2.mgr.x - c2.osd.0 - c2.osd.1 - c2.osd.2 - c2.client.0 - c2.client.1 sha1: 6fe217b2dddadf1baae6a015a57f83f4283a6078 sleep_before_teardown: 0 suite: rgw suite_branch: nautilus-py3 suite_path: /home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa suite_relpath: qa suite_repo: https://github.com/tchaikov/ceph.git suite_sha1: 2e1d102f0cb696365106419cf1e42700ba50a8ff tasks: - install: null - ceph: cluster: c1 - ceph: cluster: c2 - rgw: c1.client.0: port: 8000 valgrind: - --tool=memcheck - --max-threads=1024 c1.client.1: port: 8001 valgrind: - --tool=memcheck - --max-threads=1024 c2.client.0: port: 8000 valgrind: - --tool=memcheck - --max-threads=1024 c2.client.1: port: 8001 valgrind: - --tool=memcheck - --max-threads=1024 - rgw-multisite: null - rgw-multisite-tests: config: reconfigure_delay: 60 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.30709 2020-05-24T13:29:07.117 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa; will attempt to use it 2020-05-24T13:29:07.117 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa/tasks 2020-05-24T13:29:07.118 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-05-24T13:29:07.120 INFO:teuthology.task.internal:Checking packages... 2020-05-24T13:29:07.153 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '6fe217b2dddadf1baae6a015a57f83f4283a6078' 2020-05-24T13:29:07.153 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-05-24T13:29:07.154 INFO:teuthology.packaging:ref: None 2020-05-24T13:29:07.154 INFO:teuthology.packaging:tag: None 2020-05-24T13:29:07.155 INFO:teuthology.packaging:branch: nautilus 2020-05-24T13:29:07.155 INFO:teuthology.packaging:sha1: 6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:29:07.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=nautilus 2020-05-24T13:29:07.661 INFO:teuthology.task.internal:Found packages for ceph version 14.2.9-580.gccd9c04 2020-05-24T13:29:07.662 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-05-24T13:29:07.672 INFO:teuthology.task.internal:no buildpackages task found 2020-05-24T13:29:07.673 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-05-24T13:29:07.708 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-05-24T13:29:07.709 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-24T13:29:07.766 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:29:17.786 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:29:27.804 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:29:37.824 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:29:47.960 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:29:57.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:08.004 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:18.029 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:28.047 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:30:38.063 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:30:48.086 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:30:58.280 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:31:08.294 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:31:18.311 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:31:28.326 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-24T13:31:38.343 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-24T13:31:48.364 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_kchai@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087414'} 2020-05-24T13:31:48.387 DEBUG:teuthology.lock.ops:locked ubuntu@smithi050.front.sepia.ceph.com, ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:31:48.462 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8d833d39d8>, signals=[15]) 2020-05-24T13:31:48.674 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 7.6 2020-05-24T13:31:48.693 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of centos 7.6 2020-05-24T13:31:49.090 INFO:teuthology.orchestra.console:Power off smithi050 2020-05-24T13:31:49.092 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-05-24T13:31:49.119 INFO:teuthology.orchestra.console:Power off smithi160 2020-05-24T13:31:49.120 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-05-24T13:31:49.133 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:31:49.151 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:31:53.170 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:31:53.198 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:31:57.302 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:31:57.323 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:31:57.537 INFO:teuthology.orchestra.console:Power off for smithi160 completed 2020-05-24T13:31:57.638 INFO:teuthology.orchestra.console:Power on smithi160 2020-05-24T13:31:57.639 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-05-24T13:31:57.667 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:01.428 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:01.644 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2020-05-24T13:32:01.691 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:01.748 INFO:teuthology.orchestra.console:Power on smithi050 2020-05-24T13:32:01.749 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-05-24T13:32:01.775 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:05.802 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:05.814 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-05-24T13:32:06.015 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2020-05-24T13:32:06.043 INFO:teuthology.orchestra.console:Power on for smithi160 completed 2020-05-24T13:32:06.117 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2020-05-24T13:32:06.144 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish 2020-05-24T13:34:22.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:31.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:37.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:40.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:46.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:49.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:55.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:34:58.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:04.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:07.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:14.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:17.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:23.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:29.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:32.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:38.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:41.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:47.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:53.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:35:53.788 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-05-24T13:35:53.790 INFO:teuthology.orchestra.run.smithi050:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-05-24T13:36:09.577 INFO:teuthology.provision.fog.smithi050:Node is ready 2020-05-24T13:36:09.578 INFO:teuthology.orchestra.run.smithi050:> hostname 2020-05-24T13:36:09.703 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2020-05-24T13:36:09.704 INFO:teuthology.orchestra.run.smithi050:> grep smithi050.front.sepia.ceph.com /etc/hosts 2020-05-24T13:36:09.778 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2020-05-24T13:36:09.779 INFO:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2020-05-24T13:36:09.948 INFO:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hosts 2020-05-24T13:36:10.060 INFO:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/172.21.15.50/172.21.15.50/g' /etc/hosts 2020-05-24T13:36:10.160 INFO:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hostname 2020-05-24T13:36:10.254 INFO:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2020-05-24T13:36:10.339 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2020-05-24T13:36:10.340 DEBUG:teuthology.parallel:result is None 2020-05-24T13:36:47.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:36:47.460 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-05-24T13:36:47.520 INFO:teuthology.orchestra.run.smithi160:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-05-24T13:36:47.939 INFO:teuthology.provision.fog.smithi160:Node is ready 2020-05-24T13:36:47.940 INFO:teuthology.orchestra.run.smithi160:> hostname 2020-05-24T13:36:48.031 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160 2020-05-24T13:36:48.031 INFO:teuthology.orchestra.run.smithi160:> grep smithi160 /etc/hosts 2020-05-24T13:36:48.110 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160 2020-05-24T13:36:48.110 INFO:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2020-05-24T13:36:48.251 INFO:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160/smithi160/g' /etc/hosts 2020-05-24T13:36:48.341 INFO:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/172.21.15.160/172.21.15.160/g' /etc/hosts 2020-05-24T13:36:48.408 INFO:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160/smithi160/g' /etc/hostname 2020-05-24T13:36:48.479 INFO:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2020-05-24T13:36:48.542 INFO:teuthology.provision.fog.smithi160:Deploy complete! 2020-05-24T13:36:48.543 DEBUG:teuthology.parallel:result is None 2020-05-24T13:36:48.734 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2020-05-24T13:36:48.736 INFO:teuthology.lock.ops:New key found. Updating... 2020-05-24T13:36:48.788 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com 2020-05-24T13:36:48.789 INFO:teuthology.lock.ops:New key found. Updating... 2020-05-24T13:36:48.810 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2020-05-24T13:36:48.811 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:36:48.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:36:48.907 INFO:teuthology.orchestra.run.smithi050:> true 2020-05-24T13:36:49.321 INFO:teuthology.orchestra.run.smithi050:> uname -m 2020-05-24T13:36:49.397 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2020-05-24T13:36:49.397 INFO:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2020-05-24T13:36:49.468 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Linux" 2020-05-24T13:36:49.468 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="7 (Core)" 2020-05-24T13:36:49.468 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2020-05-24T13:36:49.469 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2020-05-24T13:36:49.469 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="7" 2020-05-24T13:36:49.469 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-05-24T13:36:49.469 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2020-05-24T13:36:49.469 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-05-24T13:36:49.470 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.centos.org/" 2020-05-24T13:36:49.470 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-05-24T13:36:49.470 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:36:49.470 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-05-24T13:36:49.470 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-05-24T13:36:49.470 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-05-24T13:36:49.471 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-05-24T13:36:49.471 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:36:49.472 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2020-05-24T13:36:49.492 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server 2020-05-24T13:36:49.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:36:49.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:36:49.590 INFO:teuthology.orchestra.run.smithi160:> true 2020-05-24T13:36:49.945 INFO:teuthology.orchestra.run.smithi160:> uname -m 2020-05-24T13:36:50.001 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2020-05-24T13:36:50.002 INFO:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2020-05-24T13:36:50.072 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Linux" 2020-05-24T13:36:50.073 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="7 (Core)" 2020-05-24T13:36:50.073 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2020-05-24T13:36:50.073 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2020-05-24T13:36:50.073 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="7" 2020-05-24T13:36:50.073 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-05-24T13:36:50.074 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2020-05-24T13:36:50.074 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-05-24T13:36:50.074 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.centos.org/" 2020-05-24T13:36:50.074 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-05-24T13:36:50.074 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:36:50.075 INFO:teuthology.orchestra.run.smithi160.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-05-24T13:36:50.075 INFO:teuthology.orchestra.run.smithi160.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-05-24T13:36:50.075 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-05-24T13:36:50.075 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-05-24T13:36:50.075 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:36:50.076 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2020-05-24T13:36:50.096 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-05-24T13:36:50.125 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0YXVafo/xd3CdpgUWNATDXI81GKJ0ZICl+CwvHbakJEQbr4WNtjRDwwJfd6XQoW6raB3VIgqfoaPfxPNVsKHwentf6qQ3k3a9713qUVRkm/qJ0op2SHVSl1j6I9kpB66p1Qjlun9XbSOprdByFbBEIgbcSGw1S2W4YKJIJnAYhSR/gQZQJPqH2giCCJv34D63vfh4BvExEygU8Y0IITtnqPwLFugXEFfLAeCgZ7/LDiWtpaHOTbVgfiO320qWw5dx7l2CEiqxmJdjJrcS67+2TQY2kicRH8qWiHhWh3JueRZldMXP1X3P7eqPY75SK8M5uYY8Guc5frCm1TcMSMlH smithi160.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbjOIgGz2r4F2rEiE18SEix1zO1uILNCLKpnyWJ3HF/upvHd69PmvXN8b1Mw3h417K/U2qzhX+yBYW9GZpuDcSi6fAoI2ripRRrYZX4yj3NyYuflpwlfxCIR/WHOH3Q5hdHfRy3L9GeA6/M7uTsHu8lFv1+bzh1pkekIr97BNdu8SkZO4Jg/VCyA+iGX/Y/YkxSJkJ1ZeE2FQgXslTKRNRBzvCoheXa1nvETZL0EdJr1PqP8RQwpK5LCnuMutlWJYU8nvf4wTLhcv/7TpJyohyK8CCAMqMRSBUAt6jho5DfLfPm/HeCwzOPkZp3/qAr5cQCNujg7WMmgl2zl556cX3 2020-05-24T13:36:50.126 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-24T13:36:50.174 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-05-24T13:36:50.183 INFO:teuthology.task.internal:Saving configuration 2020-05-24T13:36:50.198 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-05-24T13:36:50.209 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-05-24T13:36:50.227 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-05-24 13:31:48.370796', 'description': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087414', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDvB0UcmPifnDLei1NAemqTUGRcF6eyZrKMI5ZLK3pVOEZ+oCgcEJlsT90IZoxPA/36zBAhlLxzXhddBNgTuFyI=', 'os_version': '7', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi050.front.sepia.ceph.com'} 2020-05-24T13:36:50.242 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-05-24 13:31:48.370816', 'description': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087414', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDFJcfHRovLFon1SiKlEP/B92dSEqA5JZMTuzBenBqBXpUOhZXVXMFMHYprplko+WJwutoPfcRJCcDlrtprFQo=', 'os_version': '7', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi160.front.sepia.ceph.com'} 2020-05-24T13:36:50.242 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-05-24T13:36:50.252 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2020-05-24T13:36:50.252 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2020-05-24T13:36:50.253 INFO:teuthology.run_tasks:Running task console_log... 2020-05-24T13:36:50.318 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8d86af39d8>, signals=[15]) 2020-05-24T13:36:50.319 INFO:teuthology.run_tasks:Running task internal.connect... 2020-05-24T13:36:50.335 INFO:teuthology.task.internal:Opening connections... 2020-05-24T13:36:50.336 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:36:50.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:36:50.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:36:50.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:36:50.547 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-05-24T13:36:50.558 INFO:teuthology.orchestra.run.smithi050:> uname -m 2020-05-24T13:36:50.770 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2020-05-24T13:36:50.771 INFO:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2020-05-24T13:36:50.844 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Linux" 2020-05-24T13:36:50.845 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="7 (Core)" 2020-05-24T13:36:50.845 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2020-05-24T13:36:50.845 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2020-05-24T13:36:50.846 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="7" 2020-05-24T13:36:50.846 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-05-24T13:36:50.846 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2020-05-24T13:36:50.847 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-05-24T13:36:50.847 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.centos.org/" 2020-05-24T13:36:50.847 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-05-24T13:36:50.848 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:36:50.848 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-05-24T13:36:50.848 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-05-24T13:36:50.849 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-05-24T13:36:50.849 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-05-24T13:36:50.850 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:36:50.850 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2020-05-24T13:36:50.873 INFO:teuthology.orchestra.run.smithi160:> uname -m 2020-05-24T13:36:50.970 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2020-05-24T13:36:50.971 INFO:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2020-05-24T13:36:51.053 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Linux" 2020-05-24T13:36:51.053 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="7 (Core)" 2020-05-24T13:36:51.053 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2020-05-24T13:36:51.053 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2020-05-24T13:36:51.054 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="7" 2020-05-24T13:36:51.054 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-05-24T13:36:51.054 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2020-05-24T13:36:51.054 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-05-24T13:36:51.054 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.centos.org/" 2020-05-24T13:36:51.055 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-05-24T13:36:51.055 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:36:51.055 INFO:teuthology.orchestra.run.smithi160.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-05-24T13:36:51.055 INFO:teuthology.orchestra.run.smithi160.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-05-24T13:36:51.055 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-05-24T13:36:51.055 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-05-24T13:36:51.056 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:36:51.056 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2020-05-24T13:36:51.075 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-05-24T13:36:51.095 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-05-24T13:36:51.106 INFO:teuthology.task.internal:Checking for old test directory... 2020-05-24T13:36:51.107 INFO:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2020-05-24T13:36:51.111 INFO:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2020-05-24T13:36:51.196 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-05-24T13:36:51.206 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-05-24T13:36:51.207 INFO:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2020-05-24T13:36:51.211 INFO:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2020-05-24T13:36:51.307 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-05-24T13:36:51.355 INFO:teuthology.run_tasks:Running task kernel... 2020-05-24T13:36:51.376 DEBUG:teuthology.task.kernel:normalized config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c1.client.0': {'kdb': True, 'sha1': 'distro'}, 'c1.client.1': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c2.client.0': {'kdb': True, 'sha1': 'distro'}, 'c2.client.1': {'kdb': True, 'sha1': 'distro'}} 2020-05-24T13:36:51.377 INFO:teuthology.task.kernel:config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-05-24T13:36:51.378 INFO:teuthology.orchestra.run.smithi050:> uname -r 2020-05-24T13:36:51.463 INFO:teuthology.orchestra.run.smithi050.stdout:3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:36:51.464 INFO:teuthology.task.kernel:Running kernel on smithi050: 3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:36:51.464 INFO:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2020-05-24T13:36:52.402 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:36:52.525 INFO:teuthology.orchestra.run.smithi050.stdout:Determining fastest mirrors 2020-05-24T13:36:53.226 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:36:53.236 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:36:53.236 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:36:53.237 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:36:58.664 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:36:58.664 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:36:58.665 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package kernel.x86_64 0:3.10.0-1127.8.2.el7 will be installed 2020-05-24T13:36:59.939 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:37:00.910 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:37:00.911 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:37:00.912 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:37:00.912 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:37:00.912 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2020-05-24T13:37:00.912 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:37:00.912 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:37:00.913 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 3.10.0-1127.8.2.el7 updates 50 M 2020-05-24T13:37:00.913 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:37:00.913 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:37:00.913 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:37:00.913 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2020-05-24T13:37:00.914 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:37:00.914 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 50 M 2020-05-24T13:37:00.915 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 64 M 2020-05-24T13:37:00.915 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:37:00.915 INFO:teuthology.orchestra.run.smithi050.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-05-24T13:37:02.378 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:37:02.581 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:37:03.400 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:37:03.401 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:37:44.012 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-3.10.0-1127.8.2.el7.x86_64 1/1 2020-05-24T13:37:44.660 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-3.10.0-1127.8.2.el7.x86_64 1/1 2020-05-24T13:37:44.661 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:37:44.661 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:37:44.661 INFO:teuthology.orchestra.run.smithi050.stdout: kernel.x86_64 0:3.10.0-1127.8.2.el7 2020-05-24T13:37:44.661 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:37:44.662 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:37:44.713 INFO:teuthology.orchestra.run.smithi050:> rpm -q kernel --last | head -n 1 2020-05-24T13:37:45.123 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-3.10.0-1127.8.2.el7.x86_64 Sun 24 May 2020 01:37:06 PM UTC 2020-05-24T13:37:45.126 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:37:45.126 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.12.1.el7.x86_64 Expected: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:37:45.127 INFO:teuthology.orchestra.run.smithi160:> uname -r 2020-05-24T13:37:45.241 INFO:teuthology.orchestra.run.smithi160.stdout:3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:37:45.242 INFO:teuthology.task.kernel:Running kernel on smithi160: 3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:37:45.242 INFO:teuthology.orchestra.run.smithi160:> sudo yum install -y kernel 2020-05-24T13:37:46.104 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:37:46.213 INFO:teuthology.orchestra.run.smithi160.stdout:Determining fastest mirrors 2020-05-24T13:37:46.865 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:37:46.874 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:37:46.874 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:37:46.875 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:37:52.194 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:37:52.195 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:37:52.195 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package kernel.x86_64 0:3.10.0-1127.8.2.el7 will be installed 2020-05-24T13:37:53.467 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:37:54.318 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:37:54.319 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:37:54.322 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:37:54.322 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:37:54.323 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2020-05-24T13:37:54.323 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:37:54.323 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:37:54.324 INFO:teuthology.orchestra.run.smithi160.stdout: kernel x86_64 3.10.0-1127.8.2.el7 updates 50 M 2020-05-24T13:37:54.324 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:37:54.324 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:37:54.325 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:37:54.325 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2020-05-24T13:37:54.325 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:37:54.326 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 50 M 2020-05-24T13:37:54.326 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 64 M 2020-05-24T13:37:54.326 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:37:54.327 INFO:teuthology.orchestra.run.smithi160.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-05-24T13:37:55.645 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:37:55.849 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:37:56.848 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:37:56.849 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:38:35.244 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-3.10.0-1127.8.2.el7.x86_64 1/1 2020-05-24T13:38:35.824 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-3.10.0-1127.8.2.el7.x86_64 1/1 2020-05-24T13:38:35.824 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:38:35.824 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:38:35.824 INFO:teuthology.orchestra.run.smithi160.stdout: kernel.x86_64 0:3.10.0-1127.8.2.el7 2020-05-24T13:38:35.824 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:38:35.825 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:38:35.879 INFO:teuthology.orchestra.run.smithi160:> rpm -q kernel --last | head -n 1 2020-05-24T13:38:36.275 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-3.10.0-1127.8.2.el7.x86_64 Sun 24 May 2020 01:37:59 PM UTC 2020-05-24T13:38:36.277 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:36.278 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.12.1.el7.x86_64 Expected: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:36.278 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2020-05-24T13:38:36.278 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-05-24T13:38:36.279 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-05-24T13:38:36.279 INFO:teuthology.task.kernel:Installing distro kernel on c1.mon.a... 2020-05-24T13:38:36.279 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2020-05-24T13:38:36.280 INFO:teuthology.orchestra.run.smithi050:> rpm -q kernel --last | head -n 1 2020-05-24T13:38:36.649 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-3.10.0-1127.8.2.el7.x86_64 Sun 24 May 2020 01:37:06 PM UTC 2020-05-24T13:38:36.650 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:36.651 INFO:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2020-05-24T13:38:36.809 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2020-05-24T13:38:36.809 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2020-05-24T13:38:36.809 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.02 2020-05-24T13:38:36.810 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 0.76.el7.centos 2020-05-24T13:38:36.810 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2020-05-24T13:38:36.810 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Tue 11 Feb 2020 03:34:40 PM UTC 2020-05-24T13:38:36.810 INFO:teuthology.orchestra.run.smithi050.stdout:Group : System Environment/Base 2020-05-24T13:38:36.810 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 10080361 2020-05-24T13:38:36.811 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2020-05-24T13:38:36.811 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2020-05-24T13:38:36.811 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2020-05-24T13:38:36.811 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2020-05-24T13:38:36.811 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : kbuilder.bsys.centos.org 2020-05-24T13:38:36.812 INFO:teuthology.orchestra.run.smithi050.stdout:Relocations : (not relocatable) 2020-05-24T13:38:36.812 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : CentOS BuildSystem 2020-05-24T13:38:36.812 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2020-05-24T13:38:36.812 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2020-05-24T13:38:36.812 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2020-05-24T13:38:36.812 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2020-05-24T13:38:36.813 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:38:36.813 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2020-05-24T13:38:36.813 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2020-05-24T13:38:36.813 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2020-05-24T13:38:36.813 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2020-05-24T13:38:36.814 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:38:36.814 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2020-05-24T13:38:36.815 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2020-05-24T13:38:36.815 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:36.815 INFO:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2020-05-24T13:38:37.288 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2020-05-24T13:38:37.424 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:37.440 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initramfs-3.10.0-1127.8.2.el7.x86_64.img 2020-05-24T13:38:37.743 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:38:37.772 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img 2020-05-24T13:38:37.815 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2020-05-24T13:38:37.833 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2020-05-24T13:38:37.869 INFO:teuthology.orchestra.run.smithi050.stderr:Found linux image: /boot/vmlinuz-0-rescue-11867a3080154962a87601c586b1198d 2020-05-24T13:38:37.881 INFO:teuthology.orchestra.run.smithi050.stderr:Found initrd image: /boot/initramfs-0-rescue-11867a3080154962a87601c586b1198d.img 2020-05-24T13:38:39.046 INFO:teuthology.orchestra.run.smithi050.stderr:done 2020-05-24T13:38:39.048 INFO:teuthology.orchestra.run.smithi050:> mktemp 2020-05-24T13:38:39.106 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.2qmOUMAniD 2020-05-24T13:38:39.106 INFO:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2qmOUMAniD 2020-05-24T13:38:39.179 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.2qmOUMAniD 2020-05-24T13:38:39.309 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.2qmOUMAniD is 6KB 2020-05-24T13:38:39.386 INFO:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.2qmOUMAniD 2020-05-24T13:38:39.453 INFO:teuthology.orchestra.run.smithi050:> sudo grub2-set-default 0 2020-05-24T13:38:39.637 INFO:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2020-05-24T13:38:39.640 INFO:teuthology.task.kernel:Installing distro kernel on c2.mon.a... 2020-05-24T13:38:39.641 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi160.front.sepia.ceph.com, path=None, version=distro) 2020-05-24T13:38:39.641 INFO:teuthology.orchestra.run.smithi160:> rpm -q kernel --last | head -n 1 2020-05-24T13:38:40.008 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-3.10.0-1127.8.2.el7.x86_64 Sun 24 May 2020 01:37:59 PM UTC 2020-05-24T13:38:40.041 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:40.042 INFO:teuthology.orchestra.run.smithi160:> sudo rpm -qi grub2-tools 2020-05-24T13:38:40.187 INFO:teuthology.orchestra.run.smithi160.stdout:Name : grub2-tools 2020-05-24T13:38:40.187 INFO:teuthology.orchestra.run.smithi160.stdout:Epoch : 1 2020-05-24T13:38:40.188 INFO:teuthology.orchestra.run.smithi160.stdout:Version : 2.02 2020-05-24T13:38:40.188 INFO:teuthology.orchestra.run.smithi160.stdout:Release : 0.76.el7.centos 2020-05-24T13:38:40.188 INFO:teuthology.orchestra.run.smithi160.stdout:Architecture: x86_64 2020-05-24T13:38:40.188 INFO:teuthology.orchestra.run.smithi160.stdout:Install Date: Tue 11 Feb 2020 03:34:40 PM UTC 2020-05-24T13:38:40.188 INFO:teuthology.orchestra.run.smithi160.stdout:Group : System Environment/Base 2020-05-24T13:38:40.189 INFO:teuthology.orchestra.run.smithi160.stdout:Size : 10080361 2020-05-24T13:38:40.189 INFO:teuthology.orchestra.run.smithi160.stdout:License : GPLv3+ 2020-05-24T13:38:40.189 INFO:teuthology.orchestra.run.smithi160.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2020-05-24T13:38:40.189 INFO:teuthology.orchestra.run.smithi160.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2020-05-24T13:38:40.189 INFO:teuthology.orchestra.run.smithi160.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2020-05-24T13:38:40.190 INFO:teuthology.orchestra.run.smithi160.stdout:Build Host : kbuilder.bsys.centos.org 2020-05-24T13:38:40.190 INFO:teuthology.orchestra.run.smithi160.stdout:Relocations : (not relocatable) 2020-05-24T13:38:40.190 INFO:teuthology.orchestra.run.smithi160.stdout:Packager : CentOS BuildSystem 2020-05-24T13:38:40.190 INFO:teuthology.orchestra.run.smithi160.stdout:Vendor : CentOS 2020-05-24T13:38:40.190 INFO:teuthology.orchestra.run.smithi160.stdout:URL : http://www.gnu.org/software/grub/ 2020-05-24T13:38:40.190 INFO:teuthology.orchestra.run.smithi160.stdout:Summary : Support tools for GRUB. 2020-05-24T13:38:40.191 INFO:teuthology.orchestra.run.smithi160.stdout:Description : 2020-05-24T13:38:40.191 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:38:40.191 INFO:teuthology.orchestra.run.smithi160.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2020-05-24T13:38:40.191 INFO:teuthology.orchestra.run.smithi160.stdout:customizable bootloader with modular architecture. It supports a rich 2020-05-24T13:38:40.191 INFO:teuthology.orchestra.run.smithi160.stdout:variety of kernel formats, file systems, computer architectures and 2020-05-24T13:38:40.191 INFO:teuthology.orchestra.run.smithi160.stdout:hardware devices. 2020-05-24T13:38:40.192 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:38:40.192 INFO:teuthology.orchestra.run.smithi160.stdout:This subpackage provides tools for support of all platforms. 2020-05-24T13:38:40.193 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2020-05-24T13:38:40.197 INFO:teuthology.task.kernel:Updating grub on smithi160 to boot 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:40.199 INFO:teuthology.orchestra.run.smithi160:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2020-05-24T13:38:40.687 INFO:teuthology.orchestra.run.smithi160.stderr:Generating grub configuration file ... 2020-05-24T13:38:40.818 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:38:40.834 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initramfs-3.10.0-1127.8.2.el7.x86_64.img 2020-05-24T13:38:41.082 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 2020-05-24T13:38:41.104 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img 2020-05-24T13:38:41.135 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2020-05-24T13:38:41.157 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2020-05-24T13:38:41.184 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-0-rescue-11867a3080154962a87601c586b1198d 2020-05-24T13:38:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initramfs-0-rescue-11867a3080154962a87601c586b1198d.img 2020-05-24T13:38:42.302 INFO:teuthology.orchestra.run.smithi160.stderr:done 2020-05-24T13:38:42.304 INFO:teuthology.orchestra.run.smithi160:> mktemp 2020-05-24T13:38:42.358 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.gTZYa02U28 2020-05-24T13:38:42.359 INFO:teuthology.orchestra.run.smithi160:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gTZYa02U28 2020-05-24T13:38:42.423 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.gTZYa02U28 2020-05-24T13:38:42.571 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.gTZYa02U28 is 6KB 2020-05-24T13:38:42.649 INFO:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.gTZYa02U28 2020-05-24T13:38:42.709 INFO:teuthology.orchestra.run.smithi160:> sudo grub2-set-default 0 2020-05-24T13:38:42.949 INFO:teuthology.orchestra.run.smithi160:> sudo shutdown -r now 2020-05-24T13:39:12.972 INFO:teuthology.misc:Re-opening connections... 2020-05-24T13:39:12.973 INFO:teuthology.misc:trying to connect to ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:39:12.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:39:12.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:39:31.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2020-05-24T13:39:31.424 INFO:teuthology.misc:trying to connect to ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:39:31.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:39:31.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:39:49.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2020-05-24T13:39:49.855 DEBUG:teuthology.misc:waited 36.88244271278381 2020-05-24T13:39:50.856 INFO:teuthology.misc:trying to connect to ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:39:50.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:39:50.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:40:06.242 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2020-05-24T13:40:06.243 INFO:teuthology.misc:trying to connect to ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:40:06.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:40:06.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:40:24.671 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2020-05-24T13:40:24.672 DEBUG:teuthology.misc:waited 71.69914937019348 2020-05-24T13:40:25.673 INFO:teuthology.misc:trying to connect to ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:40:25.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:40:25.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:40:25.788 INFO:teuthology.orchestra.run.smithi050:> true 2020-05-24T13:40:26.192 DEBUG:teuthology.misc:waited 73.2193968296051 2020-05-24T13:40:27.194 INFO:teuthology.misc:trying to connect to ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:40:27.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:40:27.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:40:27.287 INFO:teuthology.orchestra.run.smithi160:> true 2020-05-24T13:40:27.644 DEBUG:teuthology.misc:waited 74.67110395431519 2020-05-24T13:40:28.646 INFO:teuthology.task.kernel:Checking client c1.mon.a for new kernel version... 2020-05-24T13:40:28.646 INFO:teuthology.task.kernel:Checking kernel version of c1.mon.a, want "3.10.0-1127.8.2.el7.x86_64"... 2020-05-24T13:40:28.647 INFO:teuthology.orchestra.run.smithi050:> uname -r 2020-05-24T13:40:28.754 INFO:teuthology.orchestra.run.smithi050.stdout:3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:28.755 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1127.8.2.el7.x86_64 vs 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:28.755 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-05-24T13:40:28.755 INFO:teuthology.task.kernel:Checking client c2.mon.a for new kernel version... 2020-05-24T13:40:28.755 INFO:teuthology.task.kernel:Checking kernel version of c2.mon.a, want "3.10.0-1127.8.2.el7.x86_64"... 2020-05-24T13:40:28.756 INFO:teuthology.orchestra.run.smithi160:> uname -r 2020-05-24T13:40:28.820 INFO:teuthology.orchestra.run.smithi160.stdout:3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:28.820 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1127.8.2.el7.x86_64 vs 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:28.821 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-05-24T13:40:29.821 INFO:teuthology.task.kernel:Enabling kdb on c1.mon.a... 2020-05-24T13:40:29.822 INFO:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-24T13:40:29.980 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2020-05-24T13:40:29.995 INFO:teuthology.task.kernel:Enabling kdb on c2.mon.a... 2020-05-24T13:40:29.996 INFO:teuthology.orchestra.run.smithi160:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-24T13:40:30.069 INFO:teuthology.orchestra.run.smithi160.stdout:ttyS1 2020-05-24T13:40:30.082 INFO:teuthology.run_tasks:Running task internal.base... 2020-05-24T13:40:30.102 INFO:teuthology.task.internal:Creating test directory... 2020-05-24T13:40:30.102 INFO:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-05-24T13:40:30.107 INFO:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-05-24T13:40:30.195 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-05-24T13:40:30.207 INFO:teuthology.run_tasks:Running task internal.archive... 2020-05-24T13:40:30.220 INFO:teuthology.task.internal:Creating archive directory... 2020-05-24T13:40:30.220 INFO:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-05-24T13:40:30.224 INFO:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-05-24T13:40:30.392 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-05-24T13:40:30.403 INFO:teuthology.task.internal:Enabling coredump saving... 2020-05-24T13:40:30.404 INFO:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-05-24T13:40:30.410 INFO:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-05-24T13:40:30.470 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-24T13:40:30.482 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-24T13:40:30.519 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-24T13:40:30.544 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-05-24T13:40:30.546 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-05-24T13:40:30.559 INFO:teuthology.task.internal:Configuring sudo... 2020-05-24T13:40:30.560 INFO:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-05-24T13:40:30.563 INFO:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-05-24T13:40:30.658 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-05-24T13:40:30.670 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-05-24T13:40:30.671 INFO:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-05-24T13:40:30.675 INFO:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-05-24T13:40:30.743 INFO:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-24T13:40:30.839 INFO:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-24T13:40:30.952 INFO:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-24T13:40:31.043 INFO:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-24T13:40:31.151 INFO:teuthology.orchestra.run.smithi050:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-05-24T13:40:31.236 INFO:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-24T13:40:31.299 INFO:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-05-24T13:40:31.395 INFO:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-24T13:40:31.456 INFO:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-05-24T13:40:31.533 INFO:teuthology.orchestra.run.smithi160:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-05-24T13:40:31.592 INFO:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2020-05-24T13:40:31.596 INFO:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2020-05-24T13:40:31.707 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-05-24T13:40:31.740 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-05-24T13:40:31.770 INFO:teuthology.run_tasks:Running task internal.timer... 2020-05-24T13:40:31.791 INFO:teuthology.task.internal:Starting timer... 2020-05-24T13:40:31.791 INFO:teuthology.run_tasks:Running task kernel... 2020-05-24T13:40:31.802 DEBUG:teuthology.task.kernel:normalized config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c1.client.0': {'kdb': True, 'sha1': 'distro'}, 'c1.client.1': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'c2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'c2.client.0': {'kdb': True, 'sha1': 'distro'}, 'c2.client.1': {'kdb': True, 'sha1': 'distro'}} 2020-05-24T13:40:31.802 INFO:teuthology.task.kernel:config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-05-24T13:40:31.803 INFO:teuthology.orchestra.run.smithi050:> uname -r 2020-05-24T13:40:31.897 INFO:teuthology.orchestra.run.smithi050.stdout:3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:31.897 INFO:teuthology.task.kernel:Running kernel on smithi050: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:31.898 INFO:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2020-05-24T13:40:32.830 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:40:32.944 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:40:32.967 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:40:32.973 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:40:32.974 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:40:32.974 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:40:34.130 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-3.10.0-1127.8.2.el7.x86_64 already installed and latest version 2020-05-24T13:40:34.130 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do 2020-05-24T13:40:34.150 INFO:teuthology.orchestra.run.smithi050:> echo no | sudo yum reinstall kernel || true 2020-05-24T13:40:34.414 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:40:34.572 INFO:teuthology.orchestra.run.smithi050.stderr:Skipping the running kernel: kernel-3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:34.573 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:40:34.574 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:40:34.575 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:40:34.576 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:40:34.576 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:40:34.596 INFO:teuthology.orchestra.run.smithi050.stdout:Installed package kernel-3.10.0-1062.12.1.el7.x86_64 (from updates) not available. 2020-05-24T13:40:34.596 INFO:teuthology.orchestra.run.smithi050.stdout:Installed package kernel-3.10.0-957.el7.x86_64 (from anaconda) not available. 2020-05-24T13:40:34.596 INFO:teuthology.orchestra.run.smithi050.stderr:Error: Nothing to do 2020-05-24T13:40:34.617 INFO:teuthology.task.kernel:Newest distro kernel already installed and running 2020-05-24T13:40:34.618 INFO:teuthology.orchestra.run.smithi160:> uname -r 2020-05-24T13:40:34.670 INFO:teuthology.orchestra.run.smithi160.stdout:3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:34.670 INFO:teuthology.task.kernel:Running kernel on smithi160: 3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:34.671 INFO:teuthology.orchestra.run.smithi160:> sudo yum install -y kernel 2020-05-24T13:40:35.393 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:40:35.464 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:40:35.474 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:40:35.480 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:40:35.481 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:40:35.481 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:40:36.597 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-3.10.0-1127.8.2.el7.x86_64 already installed and latest version 2020-05-24T13:40:36.597 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do 2020-05-24T13:40:36.612 INFO:teuthology.orchestra.run.smithi160:> echo no | sudo yum reinstall kernel || true 2020-05-24T13:40:36.760 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:40:36.897 INFO:teuthology.orchestra.run.smithi160.stderr:Skipping the running kernel: kernel-3.10.0-1127.8.2.el7.x86_64 2020-05-24T13:40:36.898 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:40:36.900 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:40:36.901 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:40:36.901 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:40:36.902 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:40:36.924 INFO:teuthology.orchestra.run.smithi160.stdout:Installed package kernel-3.10.0-1062.12.1.el7.x86_64 (from updates) not available. 2020-05-24T13:40:36.924 INFO:teuthology.orchestra.run.smithi160.stdout:Installed package kernel-3.10.0-957.el7.x86_64 (from anaconda) not available. 2020-05-24T13:40:36.924 INFO:teuthology.orchestra.run.smithi160.stderr:Error: Nothing to do 2020-05-24T13:40:36.943 INFO:teuthology.task.kernel:Newest distro kernel already installed and running 2020-05-24T13:40:36.944 INFO:teuthology.task.kernel:Enabling kdb on c1.mon.a... 2020-05-24T13:40:36.944 INFO:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-24T13:40:37.059 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2020-05-24T13:40:37.115 INFO:teuthology.task.kernel:Enabling kdb on c2.mon.a... 2020-05-24T13:40:37.115 INFO:teuthology.orchestra.run.smithi160:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-05-24T13:40:37.223 INFO:teuthology.orchestra.run.smithi160.stdout:ttyS1 2020-05-24T13:40:37.273 INFO:teuthology.run_tasks:Running task pcp... 2020-05-24T13:40:37.346 INFO:teuthology.run_tasks:Running task selinux... 2020-05-24T13:40:37.359 INFO:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2020-05-24T13:40:37.530 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: [ OK ] 2020-05-24T13:40:37.533 INFO:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2020-05-24T13:40:37.706 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: [ OK ] 2020-05-24T13:40:37.707 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-05-24T13:40:37.708 INFO:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2020-05-24T13:40:37.808 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2020-05-24T13:40:37.808 INFO:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2020-05-24T13:40:37.879 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2020-05-24T13:40:37.879 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi160.front.sepia.ceph.com': 'permissive'} 2020-05-24T13:40:37.880 INFO:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\)' 2020-05-24T13:40:37.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-24T13:40:38.010 INFO:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\)' 2020-05-24T13:40:38.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-05-24T13:40:38.109 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-05-24T13:40:38.109 INFO:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2020-05-24T13:40:38.228 INFO:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2020-05-24T13:40:38.336 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-05-24T13:40:38.351 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-05-24T13:40:38.352 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-05-24T13:40:38.379 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-05-24T13:40:38.383 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi050.front.sepia.ceph.com,smithi160.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-05-24T13:40:38.813 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-05-24T13:40:39.817 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-05-24T13:40:42.270 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-05-24T13:40:42.304 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:42 +0000 (0:00:00.092) 0:00:00.092 ************ 2020-05-24T13:40:42.320 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:42 +0000 (0:00:00.015) 0:00:00.107 ************ 2020-05-24T13:40:44.608 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi160.front.sepia.ceph.com] 2020-05-24T13:40:44.704 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:44 +0000 (0:00:02.384) 0:00:02.491 ************ 2020-05-24T13:40:44.815 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] 2020-05-24T13:40:44.909 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:44 +0000 (0:00:00.204) 0:00:02.696 ************ 2020-05-24T13:40:44.911 INFO:teuthology.task.ansible.out: 2020-05-24T13:40:45.260 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-05-24T13:40:45.262 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-05-24T13:40:45.298 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:45 +0000 (0:00:00.388) 0:00:03.085 ************ 2020-05-24T13:40:45.309 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-05-24T13:40:45.363 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:45 +0000 (0:00:00.065) 0:00:03.150 ************ 2020-05-24T13:40:45.982 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-05-24T13:40:45.984 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-05-24T13:40:46.031 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:46 +0000 (0:00:00.667) 0:00:03.818 ************ 2020-05-24T13:40:46.043 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-05-24T13:40:46.082 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:46 +0000 (0:00:00.051) 0:00:03.869 ************ 2020-05-24T13:40:46.090 INFO:teuthology.task.ansible.out: 2020-05-24T13:40:46.092 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-05-24T13:40:46.094 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:46.131 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:46 +0000 (0:00:00.048) 0:00:03.918 ************ 2020-05-24T13:40:46.142 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:46.189 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:46 +0000 (0:00:00.057) 0:00:03.976 ************ 2020-05-24T13:40:46.749 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-05-24T13:40:46.751 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:40:46.804 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:46 +0000 (0:00:00.615) 0:00:04.591 ************ 2020-05-24T13:40:46.821 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:40:46.856 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:46 +0000 (0:00:00.051) 0:00:04.643 ************ 2020-05-24T13:40:48.534 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-05-24T13:40:48.536 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327646.874996-269545083672296/source", "state": "file", "uid": 0} 2020-05-24T13:40:48.571 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:48 +0000 (0:00:01.713) 0:00:06.356 ************ 2020-05-24T13:40:48.578 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327646.830713-113857689601501/source", "state": "file", "uid": 0} 2020-05-24T13:40:48.614 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:48 +0000 (0:00:00.043) 0:00:06.400 ************ 2020-05-24T13:40:49.305 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtC 2020-05-24T13:40:49.307 INFO:teuthology.task.ansible.out:toHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-05-24T13:40:49.376 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-05-24T13:40:49.378 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-05-24T13:40:49.473 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-05-24T13:40:49.480 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-05-24T13:40:49.641 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:49 +0000 (0:00:01.027) 0:00:07.428 ************ 2020-05-24T13:40:49.678 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:49 +0000 (0:00:00.036) 0:00:07.464 ************ 2020-05-24T13:40:50.562 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi050.front.sepia.ceph.com] 2020-05-24T13:40:50.636 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-05-24T13:40:50.639 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:50 +0000 (0:00:00.959) 0:00:08.424 ************ 2020-05-24T13:40:50.649 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] 2020-05-24T13:40:50.729 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:50 +0000 (0:00:00.092) 0:00:08.516 ************ 2020-05-24T13:40:51.340 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** changed: [smithi050.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.169707", "end": "2020-05-24 13:40:51.312844", "rc": 0, "start": "2020-05-24 13:40:51.143137", "stderr": "", "stderr_lines": [], "stdout": "815dca2d192c9d570bf0eed74c506da390104ba6", "stdout_lines": ["815dca2d192c9d570bf0eed74c506da390104ba6"]} 2020-05-24T13:40:51.391 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:51 +0000 (0:00:00.659) 0:00:09.175 ************ 2020-05-24T13:40:51.405 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.189710", "end": "2020-05-24 13:40:51.333656", "rc": 0, "start": "2020-05-24 13:40:51.143946", "stderr": "", "stderr_lines": [], "stdout": "815dca2d192c9d570bf0eed74c506da390104ba6", "stdout_lines": ["815dca2d192c9d570bf0eed74c506da390104ba6"]} 2020-05-24T13:40:51.438 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:51 +0000 (0:00:00.049) 0:00:09.225 ************ 2020-05-24T13:40:51.731 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.016171", "end": "2020-05-24 13:40:51.700884", "failed_when_result": false, "rc": 0, "start": "2020-05-24 13:40:51.684713", "stderr": "", "stderr_lines": [], "stdout": "966e9a6bfb22958c64d4123b47ddee87a2318d5a", "stdout_lines": ["966e9a6bfb22958c64d4123b47ddee87a2318d5a"]} 2020-05-24T13:40:51.863 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:51 +0000 (0:00:00.424) 0:00:09.650 ************ 2020-05-24T13:40:51.902 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-05-24T13:40:51.904 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013070", "end": "2020-05-24 13:40:51.875411", "failed_when_result": false, "rc": 0, "start": "2020-05-24 13:40:51.862341", "stderr": "", "stderr_lines": [], "stdout": "966e9a6bfb22958c64d4123b47ddee87a2318d5a", "stdout_lines": ["966e9a6bfb22958c64d4123b47ddee87a2318d5a"]} 2020-05-24T13:40:51.946 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:51 +0000 (0:00:00.080) 0:00:09.730 ************ 2020-05-24T13:40:51.981 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:51 +0000 (0:00:00.038) 0:00:09.768 ************ 2020-05-24T13:40:51.992 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-05-24T13:40:51.996 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.037 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.050) 0:00:09.819 ************ 2020-05-24T13:40:52.043 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-05-24T13:40:52.046 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.081 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.048) 0:00:09.868 ************ 2020-05-24T13:40:52.091 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-05-24T13:40:52.093 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.133 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.052) 0:00:09.920 ************ 2020-05-24T13:40:52.142 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-05-24T13:40:52.144 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.179 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.045) 0:00:09.966 ************ 2020-05-24T13:40:52.188 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-05-24T13:40:52.190 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.226 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.046) 0:00:10.013 ************ 2020-05-24T13:40:52.238 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-05-24T13:40:52.240 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.275 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.048) 0:00:10.062 ************ 2020-05-24T13:40:52.284 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-05-24T13:40:52.286 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.323 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.048) 0:00:10.110 ************ 2020-05-24T13:40:52.333 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-05-24T13:40:52.335 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.368 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.044) 0:00:10.155 ************ 2020-05-24T13:40:52.399 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:40:52.435 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.067) 0:00:10.222 ************ 2020-05-24T13:40:52.664 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-05-24T13:40:52.666 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:40:52.703 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.267) 0:00:10.490 ************ 2020-05-24T13:40:52.739 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:40:52.787 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:40:52 +0000 (0:00:00.084) 0:00:10.574 ************ 2020-05-24T13:40:53.069 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-24T13:40:53.091 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-05-24T13:40:53.194 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-05-24T13:40:53.440 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-05-24T13:40:53.535 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-05-24T13:40:53.798 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-05-24T13:40:53.880 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-05-24T13:40:54.174 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-05-24T13:40:54.219 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-05-24T13:40:54.504 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-05-24T13:40:54.581 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-05-24T13:40:54.874 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-05-24T13:40:54.901 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-05-24T13:40:55.204 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-05-24T13:40:55.258 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-05-24T13:40:55.783 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-05-24T13:40:55.968 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-05-24T13:40:56.135 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-05-24T13:40:56.310 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-05-24T13:40:56.469 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-05-24T13:40:56.680 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-05-24T13:40:56.816 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-05-24T13:40:57.032 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-05-24T13:40:57.170 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-05-24T13:40:57.381 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-05-24T13:40:57.541 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-05-24T13:40:57.775 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-05-24T13:40:57.894 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-05-24T13:40:58.138 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-05-24T13:40:58.205 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-05-24T13:40:58.481 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-05-24T13:40:58.513 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-05-24T13:40:58.830 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-05-24T13:40:58.841 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-05-24T13:40:59.179 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1117} ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-05-24T13:40:59.529 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-05-24T13:40:59.772 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "name": "cmeno", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-05-24T13:41:00.120 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-05-24T13:41:00.152 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "name": "cmeno", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-05-24T13:41:00.460 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-05-24T13:41:00.523 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-05-24T13:41:00.798 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-05-24T13:41:00.883 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-05-24T13:41:01.153 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-05-24T13:41:01.226 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-05-24T13:41:01.501 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-05-24T13:41:01.544 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-05-24T13:41:01.819 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-05-24T13:41:01.922 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-05-24T13:41:02.139 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-05-24T13:41:02.270 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-05-24T13:41:02.468 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-05-24T13:41:02.630 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-05-24T13:41:02.813 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-05-24T13:41:02.959 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-05-24T13:41:03.153 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-05-24T13:41:03.300 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-05-24T13:41:03.479 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-05-24T13:41:03.651 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-05-24T13:41:03.794 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-05-24T13:41:03.989 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-05-24T13:41:04.135 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-05-24T13:41:04.364 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-05-24T13:41:04.460 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-05-24T13:41:04.711 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-05-24T13:41:04.830 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-05-24T13:41:05.071 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-05-24T13:41:05.192 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-05-24T13:41:05.374 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-05-24T13:41:05.538 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-05-24T13:41:05.766 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-05-24T13:41:05.906 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-05-24T13:41:06.112 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-05-24T13:41:06.247 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-05-24T13:41:06.436 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-05-24T13:41:06.569 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-05-24T13:41:06.771 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-05-24T13:41:06.924 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-24T13:41:07.162 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-05-24T13:41:07.247 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-05-24T13:41:07.517 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-24T13:41:07.582 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-05-24T13:41:07.865 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-05-24T13:41:07.878 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-05-24T13:41:08.172 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-05-24T13:41:08.226 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-05-24T13:41:08.530 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-05-24T13:41:08.591 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-05-24T13:41:08.859 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-05-24T13:41:08.973 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-05-24T13:41:09.192 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-05-24T13:41:09.329 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-05-24T13:41:09.550 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-24T13:41:09.686 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-05-24T13:41:09.894 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-05-24T13:41:10.008 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-05-24T13:41:10.224 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-05-24T13:41:10.396 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-24T13:41:10.564 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-05-24T13:41:10.736 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-05-24T13:41:10.898 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-05-24T13:41:11.087 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-05-24T13:41:11.225 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-05-24T13:41:11.447 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-05-24T13:41:11.584 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-05-24T13:41:11.820 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-05-24T13:41:11.953 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-05-24T13:41:12.203 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-05-24T13:41:12.312 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-05-24T13:41:12.554 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-05-24T13:41:12.673 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2020-05-24T13:41:12.925 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-05-24T13:41:13.032 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-05-24T13:41:13.285 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-05-24T13:41:13.381 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-05-24T13:41:13.656 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2020-05-24T13:41:13.741 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-05-24T13:41:14.013 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-05-24T13:41:14.114 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-05-24T13:41:14.370 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-05-24T13:41:14.489 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-05-24T13:41:14.698 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-05-24T13:41:14.834 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-05-24T13:41:15.058 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-05-24T13:41:15.197 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-05-24T13:41:15.401 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-05-24T13:41:15.536 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-24T13:41:15.747 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-05-24T13:41:15.892 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-05-24T13:41:16.088 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-05-24T13:41:16.255 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-05-24T13:41:16.456 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-05-24T13:41:16.550 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-05-24T13:41:16.837 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-05-24T13:41:16.906 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-05-24T13:41:17.206 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-05-24T13:41:17.272 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-05-24T13:41:17.584 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-05-24T13:41:17.641 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-05-24T13:41:17.943 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-05-24T13:41:18.006 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-05-24T13:41:18.008 INFO:teuthology.task.ansible.out: 2020-05-24T13:41:18.307 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-05-24T13:41:18.377 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-05-24T13:41:18.661 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-05-24T13:41:18.728 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-05-24T13:41:19.027 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-05-24T13:41:19.090 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-05-24T13:41:19.389 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-05-24T13:41:19.435 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-05-24T13:41:19.753 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-05-24T13:41:19.755 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-05-24T13:41:20.128 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-05-24T13:41:20.130 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-05-24T13:41:20.465 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-05-24T13:41:20.477 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-05-24T13:41:20.819 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-05-24T13:41:20.830 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-05-24T13:41:21.162 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-05-24T13:41:21.172 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-05-24T13:41:21.524 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-05-24T13:41:21.525 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-05-24T13:41:21.863 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-05-24T13:41:21.885 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-05-24T13:41:22.208 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-05-24T13:41:22.240 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-05-24T13:41:22.561 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-05-24T13:41:22.595 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-05-24T13:41:22.903 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-05-24T13:41:22.935 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-05-24T13:41:23.236 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-05-24T13:41:23.280 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-05-24T13:41:23.558 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-05-24T13:41:23.620 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-05-24T13:41:23.901 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-05-24T13:41:23.979 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-05-24T13:41:24.249 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-05-24T13:41:24.312 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-05-24T13:41:24.578 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-05-24T13:41:24.658 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-05-24T13:41:24.925 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-05-24T13:41:25.000 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-05-24T13:41:25.250 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-05-24T13:41:25.345 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-05-24T13:41:25.589 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-05-24T13:41:25.711 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-05-24T13:41:25.932 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-05-24T13:41:26.026 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-05-24T13:41:26.261 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-05-24T13:41:26.389 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-05-24T13:41:26.549 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-05-24T13:41:26.767 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-05-24T13:41:26.873 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-05-24T13:41:27.105 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-05-24T13:41:27.222 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-05-24T13:41:27.452 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-05-24T13:41:27.538 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-05-24T13:41:27.815 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-05-24T13:41:27.890 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-05-24T13:41:28.203 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-05-24T13:41:28.257 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-05-24T13:41:28.584 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-05-24T13:41:28.595 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-24T13:41:28.979 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-05-24T13:41:28.980 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-05-24T13:41:29.308 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-05-24T13:41:29.361 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-05-24T13:41:29.659 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-05-24T13:41:29.764 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-05-24T13:41:30.000 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-05-24T13:41:30.103 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-05-24T13:41:30.314 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-05-24T13:41:30.477 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-05-24T13:41:30.647 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-05-24T13:41:30.830 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-05-24T13:41:30.986 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-05-24T13:41:30.988 INFO:teuthology.task.ansible.out: 2020-05-24T13:41:31.190 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-05-24T13:41:31.324 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-05-24T13:41:31.534 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-05-24T13:41:31.864 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-05-24T13:41:31.948 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "name": "khiremat", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2020-05-24T13:41:32.208 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-05-24T13:41:32.534 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-05-24T13:41:32.628 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "name": "kapandya", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2020-05-24T13:41:32.889 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-05-24T13:41:33.149 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "name": "prasad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2020-05-24T13:41:33.451 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "name": "khiremat", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2020-05-24T13:41:33.667 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2020-05-24T13:41:34.087 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "name": "kapandya", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2020-05-24T13:41:34.299 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2020-05-24T13:41:34.678 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "name": "prasad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2020-05-24T13:41:34.932 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2020-05-24T13:41:35.064 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:35 +0000 (0:00:42.276) 0:00:52.851 ************ 2020-05-24T13:41:35.103 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-05-24T13:41:35.138 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:35 +0000 (0:00:00.073) 0:00:52.924 ************ 2020-05-24T13:41:35.200 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-05-24T13:41:35.201 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 2020-05-24T13:41:35.203 INFO:teuthology.task.ansible.out: 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAx 2020-05-24T13:41:35.205 INFO:teuthology.task.ansible.out:j+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8ea 2020-05-24T13:41:35.206 INFO:teuthology.task.ansible.out:c8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f261 2020-05-24T13:41:35.208 INFO:teuthology.task.ansible.out:02d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomai 2020-05-24T13:41:35.209 INFO:teuthology.task.ansible.out:wang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redha 2020-05-24T13:41:35.211 INFO:teuthology.task.ansible.out:t.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4Uk 2020-05-24T13:41:35.211 INFO:teuthology.task.ansible.out:ibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMiv 2020-05-24T13:41:35.214 INFO:teuthology.task.ansible.out:DNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "beng 2020-05-24T13:41:35.215 INFO:teuthology.task.ansible.out:land@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name" 2020-05-24T13:41:35.217 INFO:teuthology.task.ansible.out:: "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643 2020-05-24T13:41:35.224 INFO:teuthology.task.ansible.out:eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb 2020-05-24T13:41:35.225 INFO:teuthology.task.ansible.out:801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}]}, "changed": false} 2020-05-24T13:41:35.260 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:35 +0000 (0:00:00.122) 0:00:53.046 ************ 2020-05-24T13:41:35.270 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-24T13:41:35.272 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2020-05-24T13:41:35.856 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2020-05-24T13:41:36.103 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-05-24T13:41:36.105 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com -> localhost] => {"after": "815dca2d192c9d570bf0eed74c506da390104ba6", "attempts": 1, "before": "815dca2d192c9d570bf0eed74c506da390104ba6", "changed": false, "remote_url_changed": false} 2020-05-24T13:41:36.139 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:36 +0000 (0:00:00.879) 0:00:53.926 ************ 2020-05-24T13:41:36.520 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-05-24T13:41:36.522 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2020-05-24T13:41:36.605 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:41:36.607 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-05-24T13:41:36.639 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:36 +0000 (0:00:00.499) 0:00:54.426 ************ 2020-05-24T13:41:36.666 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-05-24T13:41:36.699 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:36 +0000 (0:00:00.059) 0:00:54.486 ************ 2020-05-24T13:41:36.751 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-05-24T13:41:36.753 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 2020-05-24T13:41:36.754 INFO:teuthology.task.ansible.out: 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAx 2020-05-24T13:41:36.759 INFO:teuthology.task.ansible.out:j+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8ea 2020-05-24T13:41:36.762 INFO:teuthology.task.ansible.out:c8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f261 2020-05-24T13:41:36.764 INFO:teuthology.task.ansible.out:02d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomai 2020-05-24T13:41:36.766 INFO:teuthology.task.ansible.out:wang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redha 2020-05-24T13:41:36.770 INFO:teuthology.task.ansible.out:t.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4Uk 2020-05-24T13:41:36.772 INFO:teuthology.task.ansible.out:ibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237 2020-05-24T13:41:36.774 INFO:teuthology.task.ansible.out:a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomth 2020-05-24T13:41:36.775 INFO:teuthology.task.ansible.out:d@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri 2020-05-24T13:41:36.777 INFO:teuthology.task.ansible.out:", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a862 2020-05-24T13:41:36.778 INFO:teuthology.task.ansible.out:89ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}]}, "changed": false} 2020-05-24T13:41:36.821 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:36 +0000 (0:00:00.115) 0:00:54.602 ************ 2020-05-24T13:41:36.930 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:41:36.935 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-05-24T13:41:37.309 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-05-24T13:41:37.507 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi050.front.sepia.ceph.com -> localhost] => {"after": "815dca2d192c9d570bf0eed74c506da390104ba6", "attempts": 1, "before": "815dca2d192c9d570bf0eed74c506da390104ba6", "changed": false, "remote_url_changed": false} 2020-05-24T13:41:37.555 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:41:37 +0000 (0:00:00.737) 0:00:55.339 ************ 2020-05-24T13:41:37.663 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:41:37.665 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqB 2020-05-24T13:41:37.667 INFO:teuthology.task.ansible.out:ZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-05-24T13:41:37.975 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-05-24T13:41:38.023 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-05-24T13:41:38.336 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-05-24T13:41:38.404 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2020-05-24T13:41:38.406 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-05-24T13:41:38.704 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-05-24T13:41:38.736 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-05-24T13:41:39.078 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqB 2020-05-24T13:41:39.080 INFO:teuthology.task.ansible.out:ZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-05-24T13:41:39.111 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-05-24T13:41:39.452 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-05-24T13:41:39.477 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-05-24T13:41:39.490 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:39.854 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2020-05-24T13:41:39.856 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-05-24T13:41:39.896 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2020-05-24T13:41:39.898 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-05-24T13:41:39.907 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:39.918 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:39.920 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:40.215 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-05-24T13:41:40.280 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-05-24T13:41:40.563 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-05-24T13:41:40.649 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2020-05-24T13:41:40.654 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-05-24T13:41:40.916 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-05-24T13:41:40.938 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:41.011 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-05-24T13:41:41.305 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2020-05-24T13:41:41.307 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-05-24T13:41:41.316 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:41.327 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:41.348 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-05-24T13:41:41.359 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:41:41.707 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-05-24T13:41:41.718 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-05-24T13:41:42.056 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff132656 2020-05-24T13:41:42.058 INFO:teuthology.task.ansible.out:1a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1g 2020-05-24T13:41:42.060 INFO:teuthology.task.ansible.out:hS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-05-24T13:41:42.409 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-05-24T13:41:42.432 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-05-24T13:41:42.744 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-05-24T13:41:42.777 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-05-24T13:41:43.110 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2020-05-24T13:41:43.112 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2020-05-24T13:41:43.146 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-05-24T13:41:43.471 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2020-05-24T13:41:43.473 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-05-24T13:41:43.514 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-05-24T13:41:43.805 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-05-24T13:41:43.869 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-05-24T13:41:44.157 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-05-24T13:41:44.211 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-05-24T13:41:44.510 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2020-05-24T13:41:44.512 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-05-24T13:41:44.591 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2020-05-24T13:41:44.593 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2020-05-24T13:41:44.859 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-05-24T13:41:44.968 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2020-05-24T13:41:44.970 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-05-24T13:41:45.232 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-05-24T13:41:45.340 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-05-24T13:41:45.578 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-05-24T13:41:45.697 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-05-24T13:41:45.948 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-05-24T13:41:46.064 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2020-05-24T13:41:46.067 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-05-24T13:41:46.277 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-05-24T13:41:46.412 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-05-24T13:41:46.618 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2020-05-24T13:41:46.620 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-05-24T13:41:46.777 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-05-24T13:41:46.966 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-05-24T13:41:47.111 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-05-24T13:41:47.310 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-05-24T13:41:47.481 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-05-24T13:41:47.574 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-05-24T13:41:47.832 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-05-24T13:41:47.926 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-05-24T13:41:48.196 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2020-05-24T13:41:48.198 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-05-24T13:41:48.279 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-05-24T13:41:48.281 INFO:teuthology.task.ansible.out: 2020-05-24T13:41:48.514 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-05-24T13:41:48.629 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-05-24T13:41:48.882 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-05-24T13:41:48.985 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-24T13:41:49.224 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-05-24T13:41:49.325 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-05-24T13:41:49.606 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-05-24T13:41:49.687 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-05-24T13:41:49.980 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-05-24T13:41:50.037 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2020-05-24T13:41:50.039 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-05-24T13:41:50.344 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-05-24T13:41:50.365 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2020-05-24T13:41:50.368 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-05-24T13:41:50.689 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-05-24T13:41:50.690 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-24T13:41:51.043 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-05-24T13:41:51.065 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-05-24T13:41:51.412 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-05-24T13:41:51.454 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-05-24T13:41:51.772 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2020-05-24T13:41:51.774 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-24T13:41:51.834 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2020-05-24T13:41:51.835 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-05-24T13:41:52.137 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2020-05-24T13:41:52.139 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2020-05-24T13:41:52.142 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-05-24T13:41:52.169 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2020-05-24T13:41:52.171 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-05-24T13:41:52.478 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-05-24T13:41:52.538 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-05-24T13:41:52.794 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-05-24T13:41:52.897 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-05-24T13:41:53.158 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-05-24T13:41:53.239 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-05-24T13:41:53.509 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-05-24T13:41:53.597 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2020-05-24T13:41:53.599 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-24T13:41:53.844 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-05-24T13:41:53.982 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2020-05-24T13:41:53.983 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2020-05-24T13:41:53.985 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-05-24T13:41:54.191 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/ 2020-05-24T13:41:54.194 INFO:teuthology.task.ansible.out:home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-05-24T13:41:54.383 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-05-24T13:41:54.528 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-05-24T13:41:54.717 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-05-24T13:41:54.878 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-05-24T13:41:55.054 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-05-24T13:41:55.215 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2020-05-24T13:41:55.217 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2020-05-24T13:41:55.432 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-05-24T13:41:55.572 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-05-24T13:41:55.786 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-05-24T13:41:55.923 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-05-24T13:41:56.144 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/ 2020-05-24T13:41:56.146 INFO:teuthology.task.ansible.out:home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-05-24T13:41:56.255 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2020-05-24T13:41:56.257 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-05-24T13:41:56.493 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-05-24T13:41:56.623 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-05-24T13:41:56.829 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-05-24T13:41:56.965 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-05-24T13:41:57.203 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2020-05-24T13:41:57.205 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2020-05-24T13:41:57.290 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-05-24T13:41:57.568 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-05-24T13:41:57.613 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-24T13:41:57.944 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-05-24T13:41:57.946 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-05-24T13:41:58.289 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-05-24T13:41:58.300 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2020-05-24T13:41:58.302 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-05-24T13:41:58.613 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-05-24T13:41:58.666 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-05-24T13:41:58.950 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-05-24T13:41:59.034 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-05-24T13:41:59.290 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-05-24T13:41:59.378 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-05-24T13:41:59.627 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-05-24T13:41:59.744 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-05-24T13:41:59.977 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-05-24T13:42:00.117 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-05-24T13:42:00.275 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-05-24T13:42:00.480 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-05-24T13:42:00.620 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-05-24T13:42:00.832 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-05-24T13:42:00.989 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-05-24T13:42:01.176 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-05-24T13:42:01.332 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-05-24T13:42:01.530 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-05-24T13:42:01.685 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-05-24T13:42:01.886 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-05-24T13:42:02.018 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-05-24T13:42:02.228 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-05-24T13:42:02.374 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-05-24T13:42:02.596 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-05-24T13:42:02.743 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-05-24T13:42:02.945 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-05-24T13:42:03.090 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-05-24T13:42:03.310 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-05-24T13:42:03.414 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-05-24T13:42:03.676 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-05-24T13:42:03.762 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-05-24T13:42:04.042 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-05-24T13:42:04.093 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-05-24T13:42:04.396 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-05-24T13:42:04.443 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-05-24T13:42:04.444 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:04.772 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-05-24T13:42:04.805 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-05-24T13:42:05.144 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2020-05-24T13:42:05.146 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-05-24T13:42:05.496 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2020-05-24T13:42:05.497 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstia 2020-05-24T13:42:05.499 INFO:teuthology.task.ansible.out:n.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-05-24T13:42:05.852 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-05-24T13:42:05.874 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-05-24T13:42:06.210 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-05-24T13:42:06.252 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-05-24T13:42:06.536 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-05-24T13:42:06.619 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-05-24T13:42:06.892 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-05-24T13:42:06.985 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-05-24T13:42:07.220 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-05-24T13:42:07.355 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-05-24T13:42:07.558 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-05-24T13:42:07.710 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2020-05-24T13:42:07.712 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-05-24T13:42:07.907 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-05-24T13:42:08.090 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2020-05-24T13:42:08.092 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-05-24T13:42:08.242 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-05-24T13:42:08.477 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-05-24T13:42:08.606 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-05-24T13:42:08.895 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-05-24T13:42:08.959 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-05-24T13:42:09.258 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-05-24T13:42:09.311 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-05-24T13:42:09.614 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-05-24T13:42:09.626 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-05-24T13:42:09.965 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-05-24T13:42:09.976 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-05-24T13:42:10.313 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-05-24T13:42:10.325 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-05-24T13:42:10.671 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-05-24T13:42:10.673 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2020-05-24T13:42:10.675 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-24T13:42:10.997 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-05-24T13:42:11.029 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-05-24T13:42:11.343 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-05-24T13:42:11.409 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-05-24T13:42:11.659 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-05-24T13:42:11.753 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-05-24T13:42:11.994 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-05-24T13:42:12.119 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-05-24T13:42:12.347 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzVa77kg0qGGE3IpMZgSYJ+UwQiDEOg3Lji470V/sQ/R7g6/v7FZpaYqGn1wG3XeUp2gGACxNKLQfGV9EGU66NvfQYqn9A8dXRpTpZuGWN8EcKK98YrSAq0s0WzeiEzVT86RHzVHIqL6/Zn4B1p9sIeY9Hb+bfKOdtYkRBDoFELtiMRzHPFaiIJFnXl+EhH8wceYqXVNNvCuOUBMnOiDYLDtujvYdPt+ff7rvXuOzbwcAKoINFM1C4FtoUEYriKlsxJ2rr7AblfDm/WIrCZcqRBU4JcudtvZOtHhvN4uL6WgzuzuhIQFyePAeIHFUn6AiwKC/MApq7z1Qj+N3pN/Nz xiubli@fedora1\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkS3qZAsTQMObkUS9QBEpYlL7A3/4ypeRiVQil7UXrZkXVhYd8RAlvSBomOHtslz6OMuvm1GtP53apQQbLb2ziJXlogua/2ju17iyBP8KfyGUzx45S3XF6GaSi3C4gyviQ0hLhgg0HSaO92qw7lbjf731hXYBWXDwkXz/6wbYecykKHTM2xeIhTqCSDIx+fdrHmYW8tXp5ZIGxmCXgLhyNLaMXlNrsGHvLixrnUb3abJebdi+diQxWn4fx2Q+WwoAnY1uwawyeFJzmLEs3qjFWmuYSKcDJ94gHHedXyDhZN1mZOs0a0E319jtXG88/SQrX3+/OypS92Hr0FF6PQwhT root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-05-24T13:42:12.494 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-05-24T13:42:12.683 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-05-24T13:42:12.877 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-05-24T13:42:13.030 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-05-24T13:42:13.245 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-05-24T13:42:13.380 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-05-24T13:42:13.597 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2020-05-24T13:42:13.599 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-05-24T13:42:13.709 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-05-24T13:42:13.964 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-05-24T13:42:14.058 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-05-24T13:42:14.310 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-05-24T13:42:14.401 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-05-24T13:42:14.654 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-05-24T13:42:14.767 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2020-05-24T13:42:15.024 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-05-24T13:42:15.105 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2020-05-24T13:42:15.382 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzVa77kg0qGGE3IpMZgSYJ+UwQiDEOg3Lji470V/sQ/R7g6/v7FZpaYqGn1wG3XeUp2gGACxNKLQfGV9EGU66NvfQYqn9A8dXRpTpZuGWN8EcKK98YrSAq0s0WzeiEzVT86RHzVHIqL6/Zn4B1p9sIeY9Hb+bfKOdtYkRBDoFELtiMRzHPFaiIJFnXl+EhH8wceYqXVNNvCuOUBMnOiDYLDtujvYdPt+ff7rvXuOzbwcAKoINFM1C4FtoUEYriKlsxJ2rr7AblfDm/WIrCZcqRBU4JcudtvZOtHhvN4uL6WgzuzuhIQFyePAeIHFUn6AiwKC/MApq7z1Qj+N3pN/Nz xiubli@fedora1\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkS3qZAsTQMObkUS9QBEpYlL7A3/4ypeRiVQil7UXrZkXVhYd8RAlvSBomOHtslz6OMuvm1GtP53apQQbLb2ziJXlogua/2ju17iyBP8KfyGUzx45S3XF6GaSi3C4gyviQ0hLhgg0HSaO92qw7lbjf731hXYBWXDwkXz/6wbYecykKHTM2xeIhTqCSDIx+fdrHmYW8tXp5ZIGxmCXgLhyNLaMXlNrsGHvLixrnUb3abJebdi+diQxWn4fx2Q+WwoAnY1uwawyeFJzmLEs3qjFWmuYSKcDJ94gHHedXyDhZN1mZOs0a0E319jtXG88/SQrX3+/OypS92Hr0FF6PQwhT root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-05-24T13:42:15.468 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2020-05-24T13:42:15.597 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:15 +0000 (0:00:38.045) 0:01:33.384 ************ 2020-05-24T13:42:15.663 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:15.664 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.686 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.688 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.698 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.700 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.710 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.742 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.745 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.766 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:15.773 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:15.774 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-05-24T13:42:16.114 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:16.116 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-05-24T13:42:16.125 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:16.173 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:16.175 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-05-24T13:42:16.452 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:16.453 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2020-05-24T13:42:16.559 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:16.561 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-05-24T13:42:16.801 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:16.803 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-05-24T13:42:16.936 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:16.938 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-05-24T13:42:17.136 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi160.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi 2020-05-24T13:42:17.138 INFO:teuthology.task.ansible.out:wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.148 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.159 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.170 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.181 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.191 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.202 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.204 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.224 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.226 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.235 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.253 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.259 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.270 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.281 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.283 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.293 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.307 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.309 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-05-24T13:42:17.310 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.321 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.323 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.333 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.344 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.355 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.364 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.366 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.376 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.387 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.398 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.400 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.410 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.420 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.431 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.433 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.442 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.453 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.482 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.484 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.503 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.505 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.516 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.527 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.538 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.540 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.550 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.551 INFO:teuthology.task.ansible.out: skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.564 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.583 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.594 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.605 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.617 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.627 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.649 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.659 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.681 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.682 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs" 2020-05-24T13:42:17.684 INFO:teuthology.task.ansible.out:: true} 2020-05-24T13:42:17.702 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.709 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.711 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.721 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.732 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.743 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.745 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:17.765 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.766 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.777 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.780 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.790 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.791 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.802 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.814 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.815 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.827 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.848 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.850 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.851 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.862 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.864 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.874 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.875 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.893 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.894 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.896 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.910 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.911 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.921 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.943 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.944 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.955 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.956 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.966 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:17.977 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.029 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:18 +0000 (0:00:02.432) 0:01:35.816 ************ 2020-05-24T13:42:18.049 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:18.054 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2020-05-24T13:42:18.061 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-05-24T13:42:18.063 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.124 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:18 +0000 (0:00:00.095) 0:01:35.911 ************ 2020-05-24T13:42:18.405 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:18.406 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2020-05-24T13:42:18.490 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:18.492 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-05-24T13:42:18.769 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-05-24T13:42:18.771 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2020-05-24T13:42:18.820 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:18.822 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-05-24T13:42:18.854 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:18 +0000 (0:00:00.729) 0:01:36.641 ************ 2020-05-24T13:42:18.910 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:18.912 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.922 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.933 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.944 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.955 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.966 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.967 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:18.977 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:19.117 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:19.119 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-05-24T13:42:19.332 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:19.334 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-05-24T13:42:19.347 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:19.467 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi160.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-05-24T13:42:19.696 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:19.698 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'aemerson', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2020-05-24T13:42:19.774 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:19.776 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-05-24T13:42:20.024 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-05-24T13:42:20.108 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:20.110 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-05-24T13:42:20.399 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:20.401 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2020-05-24T13:42:20.421 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.423 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi160.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-05-24T13:42:20.433 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-05-24T13:42:20.434 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.444 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.476 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.478 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.488 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.510 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.514 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.521 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.533 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.544 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.555 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.568 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.579 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.590 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.592 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.603 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.604 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.616 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.625 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.627 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.636 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.647 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.658 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.660 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.669 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.671 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.681 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.691 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.714 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.715 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.726 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.737 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.739 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.750 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.774 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.775 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.777 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.799 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.810 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.811 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.823 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.825 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:20.837 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.839 INFO:teuthology.task.ansible.out: skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.861 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.863 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.865 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.866 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.878 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.889 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.890 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.901 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.903 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.923 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.925 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.926 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.939 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.941 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.951 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.953 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.973 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.975 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.988 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:20.998 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.000 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.010 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.022 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.033 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:21.035 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "gmeno", "name": "gmeno", "remove": false, "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.046 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.057 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.059 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.068 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.070 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.088 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.101 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.103 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.112 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.114 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.133 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.135 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.135 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.150 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.157 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.159 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.168 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.179 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.181 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d\nxiubli@lxbceph0 s0gvBwazopcgx4LPu0pQiA 1b2012fe8c6ea0a54dfd0e7a6fb1a19982b934cf6d84a64d21aa16dbe8510db3\n"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.191 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.202 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.213 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.215 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.224 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.235 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.237 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.246 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.258 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.314 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:21 +0000 (0:00:02.459) 0:01:39.101 ************ 2020-05-24T13:42:21.347 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-05-24T13:42:21.349 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:21.410 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:21 +0000 (0:00:00.095) 0:01:39.197 ************ 2020-05-24T13:42:21.423 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:21.423 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-05-24T13:42:21.708 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-05-24T13:42:21.815 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-05-24T13:42:22.000 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-05-24T13:42:22.179 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-05-24T13:42:22.321 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-05-24T13:42:22.522 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-05-24T13:42:22.634 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-05-24T13:42:22.864 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-05-24T13:42:22.991 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-05-24T13:42:23.213 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-05-24T13:42:23.334 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-05-24T13:42:23.543 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-05-24T13:42:23.680 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-05-24T13:42:23.894 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-05-24T13:42:23.994 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-05-24T13:42:24.336 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-05-24T13:42:24.507 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "gmeno", "name": "gmeno", "remove": false, "state": "absent"} 2020-05-24T13:42:24.656 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-05-24T13:42:24.843 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-05-24T13:42:24.984 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-05-24T13:42:25.175 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-05-24T13:42:25.330 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-05-24T13:42:25.529 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-05-24T13:42:25.673 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-05-24T13:42:25.854 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-05-24T13:42:25.991 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-05-24T13:42:26.193 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-05-24T13:42:26.331 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-05-24T13:42:26.525 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-05-24T13:42:26.661 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-05-24T13:42:26.866 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-05-24T13:42:27.002 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-05-24T13:42:27.221 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-05-24T13:42:27.337 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-05-24T13:42:27.598 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-05-24T13:42:27.674 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-05-24T13:42:27.951 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-05-24T13:42:28.013 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-05-24T13:42:28.300 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-05-24T13:42:28.343 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-05-24T13:42:28.645 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-05-24T13:42:28.656 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-05-24T13:42:28.976 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-05-24T13:42:29.004 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-05-24T13:42:29.313 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-05-24T13:42:29.357 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-05-24T13:42:29.638 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-05-24T13:42:29.693 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-05-24T13:42:29.960 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-05-24T13:42:30.012 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-05-24T13:42:30.306 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-05-24T13:42:30.345 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-05-24T13:42:30.637 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-05-24T13:42:30.713 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-05-24T13:42:30.981 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-05-24T13:42:31.073 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-05-24T13:42:31.314 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-05-24T13:42:31.439 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-05-24T13:42:31.667 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-05-24T13:42:31.749 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-05-24T13:42:32.004 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-05-24T13:42:32.088 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-05-24T13:42:32.326 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-05-24T13:42:32.411 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-05-24T13:42:32.639 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-05-24T13:42:32.707 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:32 +0000 (0:00:11.296) 0:01:50.494 ************ 2020-05-24T13:42:32.740 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-05-24T13:42:33.090 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-05-24T13:42:33.413 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-05-24T13:42:33.762 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-05-24T13:42:33.863 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-05-24T13:42:33.870 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "checksum": "fd7540224fb799c0bce2cee5784635eafe11ed5f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "8ac82a894b7d17ffb3e18953ec0a949b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327752.7365582-5162333968003/source", "state": "file", "uid": 0} 2020-05-24T13:42:33.911 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:33 +0000 (0:00:01.203) 0:01:51.698 ************ 2020-05-24T13:42:33.950 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-05-24T13:42:33.952 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => { "msg": "Host smithi160.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2020-05-24T13:42:33.991 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:33 +0000 (0:00:00.080) 0:01:51.778 ************ 2020-05-24T13:42:34.038 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-05-24T13:42:34.039 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-05-24T13:42:34.075 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:34 +0000 (0:00:00.083) 0:01:51.862 ************ 2020-05-24T13:42:34.105 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-05-24T13:42:34.107 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:34.153 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:34 +0000 (0:00:00.078) 0:01:51.940 ************ 2020-05-24T13:42:34.162 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:34.163 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-05-24T13:42:34.445 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-05-24T13:42:34.601 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-05-24T13:42:34.603 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.103743", "end": "2020-05-24 13:42:34.569252", "rc": 0, "start": "2020-05-24 13:42:34.465509", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-05-24T13:42:34.650 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:34 +0000 (0:00:00.497) 0:01:52.437 ************ 2020-05-24T13:42:34.701 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:34.745 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:34 +0000 (0:00:00.094) 0:01:52.532 ************ 2020-05-24T13:42:34.747 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:34.774 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:34.775 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:34.806 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:34 +0000 (0:00:00.061) 0:01:52.593 ************ 2020-05-24T13:42:34.846 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-05-24T13:42:34.847 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:34.879 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:34 +0000 (0:00:00.072) 0:01:52.666 ************ 2020-05-24T13:42:34.918 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-05-24T13:42:34.919 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:34.955 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:34 +0000 (0:00:00.076) 0:01:52.742 ************ 2020-05-24T13:42:34.991 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-05-24T13:42:34.992 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.029 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.073) 0:01:52.816 ************ 2020-05-24T13:42:35.059 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-05-24T13:42:35.060 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.119 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.090) 0:01:52.906 ************ 2020-05-24T13:42:35.131 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:35.133 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-05-24T13:42:35.165 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi160.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi160.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:35.198 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.079) 0:01:52.985 ************ 2020-05-24T13:42:35.238 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-05-24T13:42:35.242 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.274 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.076) 0:01:53.061 ************ 2020-05-24T13:42:35.316 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.355 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.080) 0:01:53.142 ************ 2020-05-24T13:42:35.394 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi160.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:35.432 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.077) 0:01:53.219 ************ 2020-05-24T13:42:35.462 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:35.464 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.505 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.072) 0:01:53.292 ************ 2020-05-24T13:42:35.534 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-05-24T13:42:35.536 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:35.570 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.064) 0:01:53.356 ************ 2020-05-24T13:42:35.612 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-05-24T13:42:35.614 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.650 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.081) 0:01:53.437 ************ 2020-05-24T13:42:35.688 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-05-24T13:42:35.690 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.721 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.070) 0:01:53.507 ************ 2020-05-24T13:42:35.759 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-05-24T13:42:35.761 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.801 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.080) 0:01:53.588 ************ 2020-05-24T13:42:35.809 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:35.811 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-05-24T13:42:35.851 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-05-24T13:42:35.852 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:35.887 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.086) 0:01:53.674 ************ 2020-05-24T13:42:35.924 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-05-24T13:42:35.925 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:35.963 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:35 +0000 (0:00:00.076) 0:01:53.750 ************ 2020-05-24T13:42:36.002 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-24T13:42:36.004 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:36.037 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.074) 0:01:53.824 ************ 2020-05-24T13:42:36.076 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:36.117 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.072) 0:01:53.897 ************ TASK [users : Remove revoked users] ******************************************** ok: [smithi050.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-05-24T13:42:36.150 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:36.183 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.073) 0:01:53.970 ************ 2020-05-24T13:42:36.222 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-05-24T13:42:36.223 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:36.271 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.088) 0:01:54.058 ************ 2020-05-24T13:42:36.312 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-05-24T13:42:36.313 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:36.350 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.078) 0:01:54.137 ************ 2020-05-24T13:42:36.408 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-05-24T13:42:36.410 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:36.445 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.094) 0:01:54.232 ************ 2020-05-24T13:42:36.458 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-05-24T13:42:36.460 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-05-24T13:42:36.481 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-05-24T13:42:36.483 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:36.526 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.081) 0:01:54.313 ************ 2020-05-24T13:42:36.563 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.037) 0:01:54.350 ************ 2020-05-24T13:42:36.605 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-05-24T13:42:36.606 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:36.637 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.074) 0:01:54.424 ************ 2020-05-24T13:42:36.678 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:36.713 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:36 +0000 (0:00:00.075) 0:01:54.500 ************ 2020-05-24T13:42:37.073 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-05-24T13:42:37.075 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:42:37.115 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:37 +0000 (0:00:00.401) 0:01:54.902 ************ 2020-05-24T13:42:37.692 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "checksum": "fd7540224fb799c0bce2cee5784635eafe11ed5f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "8ac82a894b7d17ffb3e18953ec0a949b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327756.558406-207668462008122/source", "state": "file", "uid": 0} 2020-05-24T13:42:37.736 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:37 +0000 (0:00:00.621) 0:01:55.523 ************ 2020-05-24T13:42:37.794 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi050.front.sepia.ceph.com] => { "msg": "Host smithi050.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2020-05-24T13:42:37.840 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:37 +0000 (0:00:00.103) 0:01:55.627 ************ 2020-05-24T13:42:37.890 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi050.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-05-24T13:42:37.923 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:37 +0000 (0:00:00.083) 0:01:55.710 ************ 2020-05-24T13:42:37.965 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-05-24T13:42:37.967 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:38.009 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.085) 0:01:55.796 ************ 2020-05-24T13:42:38.278 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-05-24T13:42:38.280 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "md5sum": "5170d95301fca1f1242a0fb0a528bbcf", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327757.1679127-82307817374575/source", "state": "file", "uid": 0} 2020-05-24T13:42:38.451 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-05-24T13:42:38.453 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.098364", "end": "2020-05-24 13:42:38.423939", "rc": 0, "start": "2020-05-24 13:42:38.325575", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-05-24T13:42:38.484 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.475) 0:01:56.271 ************ 2020-05-24T13:42:38.513 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-05-24T13:42:38.515 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:38.546 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.061) 0:01:56.333 ************ 2020-05-24T13:42:38.578 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-05-24T13:42:38.580 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:38.629 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.082) 0:01:56.416 ************ 2020-05-24T13:42:38.662 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:38.693 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.063) 0:01:56.480 ************ 2020-05-24T13:42:38.727 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-05-24T13:42:38.729 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:38.761 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.068) 0:01:56.548 ************ 2020-05-24T13:42:38.790 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-05-24T13:42:38.792 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:38.827 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.065) 0:01:56.614 ************ 2020-05-24T13:42:38.867 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-05-24T13:42:38.869 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:38.904 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.077) 0:01:56.692 ************ 2020-05-24T13:42:38.953 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-05-24T13:42:38.955 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi050.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:38.987 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:38 +0000 (0:00:00.082) 0:01:56.774 ************ 2020-05-24T13:42:39.046 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.074 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.086) 0:01:56.861 ************ 2020-05-24T13:42:39.102 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-05-24T13:42:39.104 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.136 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.062) 0:01:56.923 ************ 2020-05-24T13:42:39.175 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-05-24T13:42:39.177 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:39.208 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.072) 0:01:56.995 ************ 2020-05-24T13:42:39.247 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-05-24T13:42:39.249 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.294 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.081) 0:01:57.077 ************ 2020-05-24T13:42:39.329 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-05-24T13:42:39.331 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:39.364 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.073) 0:01:57.151 ************ 2020-05-24T13:42:39.393 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-05-24T13:42:39.394 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "md5sum": "00fe4de6cbff2433b9d92afb443fe308", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327758.2899604-179263475714731/source", "state": "file", "uid": 0} 2020-05-24T13:42:39.444 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.080) 0:01:57.231 ************ 2020-05-24T13:42:39.454 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-05-24T13:42:39.456 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.496 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.052) 0:01:57.283 ************ 2020-05-24T13:42:39.537 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-05-24T13:42:39.542 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.573 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.076) 0:01:57.360 ************ 2020-05-24T13:42:39.615 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-05-24T13:42:39.616 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.655 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.082) 0:01:57.442 ************ 2020-05-24T13:42:39.706 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-05-24T13:42:39.707 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.743 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.088) 0:01:57.531 ************ 2020-05-24T13:42:39.784 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-05-24T13:42:39.785 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:39.821 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.077) 0:01:57.608 ************ 2020-05-24T13:42:39.879 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-24T13:42:39.880 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.920 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.098) 0:01:57.707 ************ 2020-05-24T13:42:39.963 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-05-24T13:42:39.965 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:39.999 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:39 +0000 (0:00:00.079) 0:01:57.786 ************ 2020-05-24T13:42:40.031 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:40.063 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:40 +0000 (0:00:00.063) 0:01:57.850 ************ 2020-05-24T13:42:40.103 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-05-24T13:42:40.105 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:40.144 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:40 +0000 (0:00:00.081) 0:01:57.931 ************ 2020-05-24T13:42:40.183 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-05-24T13:42:40.185 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:40.217 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:40 +0000 (0:00:00.072) 0:01:58.004 ************ 2020-05-24T13:42:40.271 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:40.310 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:40 +0000 (0:00:00.093) 0:01:58.097 ************ 2020-05-24T13:42:40.349 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-05-24T13:42:40.351 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-05-24T13:42:40.383 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:40 +0000 (0:00:00.073) 0:01:58.170 ************ 2020-05-24T13:42:40.430 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:40.467 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:40 +0000 (0:00:00.081) 0:01:58.252 ************ 2020-05-24T13:42:40.505 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-05-24T13:42:40.507 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:40.542 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:40 +0000 (0:00:00.076) 0:01:58.329 ************ 2020-05-24T13:42:40.634 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:40.639 INFO:teuthology.task.ansible.out:TASK [common : Configure local epel mirrorlists] ******************************* 2020-05-24T13:42:40.641 INFO:teuthology.task.ansible.out: ok: [smithi160.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "state": "file", "uid": 0} 2020-05-24T13:42:40.643 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:40.895 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi050.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:42:40.930 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:40 +0000 (0:00:00.388) 0:01:58.717 ************ 2020-05-24T13:42:41.616 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-05-24T13:42:41.618 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "state": "file", "uid": 0} 2020-05-24T13:42:41.656 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:41 +0000 (0:00:00.725) 0:01:59.443 ************ 2020-05-24T13:42:42.126 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "md5sum": "5170d95301fca1f1242a0fb0a528bbcf", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327760.9763398-73198089891433/source", "state": "file", "uid": 0} 2020-05-24T13:42:42.313 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-24T13:42:42.315 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum module rather than running 'yum'. If you need to use command because yum is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": "yum clean all", "delta": "0:00:00.323537", "end": "2020-05-24 13:42:42.276503", "rc": 0, "start": "2020-05-24 13:42:41.952966", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nCleaning up list of fastest mirrors", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Cleaning up list of fastest mirrors"]} 2020-05-24T13:42:42.348 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.692) 0:02:00.135 ************ 2020-05-24T13:42:42.391 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.446 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.094) 0:02:00.230 ************ 2020-05-24T13:42:42.513 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.558 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.111) 0:02:00.341 ************ 2020-05-24T13:42:42.583 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-05-24T13:42:42.585 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.624 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.069) 0:02:00.411 ************ 2020-05-24T13:42:42.663 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-05-24T13:42:42.665 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.704 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.080) 0:02:00.491 ************ 2020-05-24T13:42:42.737 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.764 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.060) 0:02:00.551 ************ 2020-05-24T13:42:42.802 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-05-24T13:42:42.804 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.843 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.073) 0:02:00.624 ************ 2020-05-24T13:42:42.869 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-05-24T13:42:42.871 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.910 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.072) 0:02:00.696 ************ 2020-05-24T13:42:42.949 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-05-24T13:42:42.952 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:42.990 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:42 +0000 (0:00:00.080) 0:02:00.777 ************ 2020-05-24T13:42:43.342 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "md5sum": "00fe4de6cbff2433b9d92afb443fe308", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1590327762.1327841-188910614065101/source", "state": "file", "uid": 0} 2020-05-24T13:42:43.367 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:43 +0000 (0:00:00.376) 0:02:01.154 ************ 2020-05-24T13:42:44.385 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-05-24T13:42:44.387 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "state": "file", "uid": 0} 2020-05-24T13:42:44.534 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-05-24T13:42:44.536 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.196459", "end": "2020-05-24 13:42:44.500461", "rc": 0, "start": "2020-05-24 13:42:43.304002", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:42:44.571 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:44 +0000 (0:00:01.203) 0:02:02.358 ************ 2020-05-24T13:42:45.344 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-05-24T13:42:45.346 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "state": "file", "uid": 0} 2020-05-24T13:42:45.384 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:45 +0000 (0:00:00.812) 0:02:03.171 ************ 2020-05-24T13:42:45.457 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-05-24T13:42:45.459 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2020-05-24T13:42:45.496 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:45 +0000 (0:00:00.109) 0:02:03.280 ************ 2020-05-24T13:42:45.566 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.601 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:45 +0000 (0:00:00.108) 0:02:03.388 ************ 2020-05-24T13:42:45.641 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-05-24T13:42:45.646 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.690 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:45 +0000 (0:00:00.089) 0:02:03.477 ************ 2020-05-24T13:42:45.720 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-05-24T13:42:45.722 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.757 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:45 +0000 (0:00:00.066) 0:02:03.543 ************ 2020-05-24T13:42:45.794 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-05-24T13:42:45.796 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:45.830 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:45 +0000 (0:00:00.073) 0:02:03.617 ************ 2020-05-24T13:42:46.053 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-05-24T13:42:46.055 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": "yum clean all", "delta": "0:00:00.306259", "end": "2020-05-24 13:42:46.019929", "rc": 0, "start": "2020-05-24 13:42:45.713670", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nCleaning up list of fastest mirrors", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Cleaning up list of fastest mirrors"]} 2020-05-24T13:42:46.106 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.275) 0:02:03.893 ************ 2020-05-24T13:42:46.135 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-05-24T13:42:46.142 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.170 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.064) 0:02:03.957 ************ 2020-05-24T13:42:46.201 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-05-24T13:42:46.203 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.242 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.072) 0:02:04.029 ************ 2020-05-24T13:42:46.282 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-05-24T13:42:46.284 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.315 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.072) 0:02:04.102 ************ 2020-05-24T13:42:46.344 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.383 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.067) 0:02:04.170 ************ 2020-05-24T13:42:46.418 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.451 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.068) 0:02:04.238 ************ 2020-05-24T13:42:46.492 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-05-24T13:42:46.494 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.531 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.079) 0:02:04.318 ************ 2020-05-24T13:42:46.573 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.619 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.082) 0:02:04.400 ************ 2020-05-24T13:42:46.647 INFO:teuthology.task.ansible.out: 2020-05-24T13:42:46.649 INFO:teuthology.task.ansible.out:TASK [common : Refresh repos] ************************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:46.703 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.090) 0:02:04.490 ************ 2020-05-24T13:42:46.798 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-05-24T13:42:46.800 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-05-24T13:42:46.836 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:46 +0000 (0:00:00.132) 0:02:04.623 ************ 2020-05-24T13:42:48.527 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-05-24T13:42:48.529 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.484273", "end": "2020-05-24 13:42:48.500464", "rc": 0, "start": "2020-05-24 13:42:47.016191", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:42:48.564 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:48 +0000 (0:00:01.727) 0:02:06.351 ************ 2020-05-24T13:42:49.103 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-05-24T13:42:49.104 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2020-05-24T13:42:49.136 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:49 +0000 (0:00:00.571) 0:02:06.923 ************ 2020-05-24T13:42:49.164 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:49.202 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:49 +0000 (0:00:00.066) 0:02:06.989 ************ 2020-05-24T13:42:49.232 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-05-24T13:42:49.233 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:49.267 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:49 +0000 (0:00:00.065) 0:02:07.054 ************ 2020-05-24T13:42:49.300 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-05-24T13:42:49.302 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:49.336 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:49 +0000 (0:00:00.068) 0:02:07.123 ************ 2020-05-24T13:42:49.366 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-05-24T13:42:49.368 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:42:49.405 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:49 +0000 (0:00:00.068) 0:02:07.192 ************ 2020-05-24T13:42:50.352 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-05-24T13:42:50.354 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-05-24T13:42:50.392 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:42:50 +0000 (0:00:00.986) 0:02:08.179 ************ 2020-05-24T13:43:01.730 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-05-24T13:43:01.732 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["smartmontools", "1:7.0-2.el7.x86_64 from base"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package smartmontools.x86_64 1:7.0-1.el7_7.1 will be updated\n---> Package smartmontools.x86_64 1:7.0-2.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n smartmontools x86_64 1:7.0-2.el7 base 546 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package\n\nTotal download size: 546 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : 1:smartmontools-7.0-2.el7.x86_64 1/2 \n Cleanup : 1:smartmontools-7.0-1.el7_7.1.x86_64 2/2 \n Verifying : 1:smartmontools-7.0-2.el7.x86_64 1/2 \n Verifying : 1:smartmontools-7.0-1.el7_7.1.x86_64 2/2 \n\nUpdated:\n smartmontools.x86_64 1:7.0-2.el7 \n\nComplete!\n"]} 2020-05-24T13:43:01.768 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:01 +0000 (0:00:11.375) 0:02:19.555 ************ 2020-05-24T13:43:02.736 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi160.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-05-24T13:43:03.660 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-05-24T13:43:04.071 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-05-24T13:43:04.073 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["smartmontools", "1:7.0-2.el7.x86_64 from base"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package smartmontools.x86_64 1:7.0-1.el7_7.1 will be updated\n---> Package smartmontools.x86_64 1:7.0-2.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n smartmontools x86_64 1:7.0-2.el7 base 546 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package\n\nTotal download size: 546 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : 1:smartmontools-7.0-2.el7.x86_64 1/2 \n Cleanup : 1:smartmontools-7.0-1.el7_7.1.x86_64 2/2 \n Verifying : 1:smartmontools-7.0-2.el7.x86_64 1/2 \n Verifying : 1:smartmontools-7.0-1.el7_7.1.x86_64 2/2 \n\nUpdated:\n smartmontools.x86_64 1:7.0-2.el7 \n\nComplete!\n"]} 2020-05-24T13:43:04.112 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:02.343) 0:02:21.898 ************ 2020-05-24T13:43:04.483 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-05-24T13:43:04.485 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-05-24T13:43:04.525 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.414) 0:02:22.312 ************ 2020-05-24T13:43:04.828 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-05-24T13:43:04.830 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:43:04.862 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:04 +0000 (0:00:00.336) 0:02:22.649 ************ 2020-05-24T13:43:05.102 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-05-24T13:43:05.103 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-05-24T13:43:05.749 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-05-24T13:43:05.750 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-05-24T13:43:06.024 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-05-24T13:43:06.026 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-05-24T13:43:06.600 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-05-24T13:43:06.602 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-05-24T13:43:07.002 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-05-24T13:43:07.004 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-05-24T13:43:07.051 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:07 +0000 (0:00:02.189) 0:02:24.838 ************ 2020-05-24T13:43:07.376 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-05-24T13:43:07.378 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:43:07.423 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:07 +0000 (0:00:00.371) 0:02:25.210 ************ 2020-05-24T13:43:07.478 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi160.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-05-24T13:43:07.522 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:07 +0000 (0:00:00.098) 0:02:25.309 ************ 2020-05-24T13:43:07.573 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:07.607 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:07 +0000 (0:00:00.085) 0:02:25.394 ************ 2020-05-24T13:43:07.667 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-05-24T13:43:07.716 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:07 +0000 (0:00:00.109) 0:02:25.503 ************ 2020-05-24T13:43:08.051 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi160.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-05-24T13:43:08.092 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:08 +0000 (0:00:00.375) 0:02:25.879 ************ 2020-05-24T13:43:08.159 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi160.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:08.165 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:08.176 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:08.187 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:08.223 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:08 +0000 (0:00:00.131) 0:02:26.010 ************ 2020-05-24T13:43:08.375 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-05-24T13:43:08.377 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-05-24T13:43:09.342 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-05-24T13:43:10.277 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-05-24T13:43:10.315 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:10 +0000 (0:00:02.092) 0:02:28.102 ************ 2020-05-24T13:43:10.351 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-05-24T13:43:10.353 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:10.386 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:10 +0000 (0:00:00.070) 0:02:28.173 ************ 2020-05-24T13:43:10.437 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-05-24T13:43:10.439 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-05-24T13:43:10.475 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:10 +0000 (0:00:00.088) 0:02:28.262 ************ 2020-05-24T13:43:10.798 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi050.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-05-24T13:43:10.854 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:10 +0000 (0:00:00.371) 0:02:28.633 ************ 2020-05-24T13:43:10.904 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi050.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:10.915 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:10.917 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:10.927 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:10.962 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:10 +0000 (0:00:00.115) 0:02:28.749 ************ 2020-05-24T13:43:16.176 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["nagios-plugins", "2.3.3-2.el7.x86_64 from epel"], ["nrpe", "4.0.3-1.el7.x86_64 from epel"], ["nagios-common", "4.4.5-7.el7.x86_64 from epel"], ["nagios-plugins-load", "2.3.3-2.el7.x86_64 from epel"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package nagios-common.x86_64 0:4.4.3-1.el7 will be updated\n---> Package nagios-common.x86_64 0:4.4.5-7.el7 will be an update\n---> Package nagios-plugins.x86_64 0:2.3.1-1.el7 will be updated\n---> Package nagios-plugins.x86_64 0:2.3.3-2.el7 will be an update\n---> Package nagios-plugins-load.x86_64 0:2.3.1-1.el7 will be updated\n---> Package nagios-plugins-load.x86_64 0:2.3.3-2.el7 will be an update\n---> Package nrpe.x86_64 0:3.2.1-8.el7 will be updated\n---> Package nrpe.x86_64 0:4.0.3-1.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n nagios-common x86_64 4.4.5-7.el7 epel 27 k\n nagios-plugins x86_64 2.3.3-2.el7 epel 113 k\n nagios-plugins-load x86_64 2.3.3-2.el7 epel 41 k\n nrpe x86_64 4.0.3-1.el7 epel 264 k\n\nTransaction Sum 2020-05-24T13:43:16.186 INFO:teuthology.task.ansible.out:mary\n================================================================================\nUpgrade 4 Packages\n\nTotal download size: 446 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 3.4 MB/s | 446 kB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : nagios-common-4.4.5-7.el7.x86_64 1/8 \n Updating : nagios-plugins-2.3.3-2.el7.x86_64 2/8 \n Updating : nagios-plugins-load-2.3.3-2.el7.x86_64 3/8 \n Updating : nrpe-4.0.3-1.el7.x86_64 4/8 \nwarning: /etc/nagios/nrpe.cfg created as /etc/nagios/nrpe.cfg.rpmnew\n Cleanup : nagios-plugins-load-2.3.1-1.el7.x86_64 5/8 \n Cleanup : nagios-plugins-2.3.1-1.el7.x86_64 6/8 \n Cleanup : nrpe-3.2.1-8.el7.x86_64 7/8 \n Cleanup : nagios-common-4.4.3-1.el7.x86_64 8/8 \n Verifying : nrpe-4.0.3-1.el7.x86_64 1/8 \n Verifying : nagios-plugins-2.3.3-2.el7.x86_64 2/8 \n Verifying : nagios-plugins-load-2.3.3-2.el7.x86_64 3/8 \n Verifying : nagios-common-4.4.5-7.el7.x86_64 4/8 \n Verifying : nrpe-3.2.1-8.el7.x86_64 5/8 \n Verifying : nagios-plugins-2.3.1-1.el7.x86_64 6/8 \n Verifying : nagios-plugins-load-2.3.1-1.el7.x86_64 7/8 \n Verifying : nagios-common-4.4.3-1.el7.x86_64 8/8 \n\nUpdated:\n nagios-common.x86_64 0:4.4.5-7.el7 nagios-plugins.x86_64 0:2.3.3-2.el7 \n nagios-plugins-load.x86_64 2020-05-24T13:43:16.188 INFO:teuthology.task.ansible.out:0:2.3.3-2.el7 nrpe.x86_64 0:4.0.3-1.el7 \n\nComplete!\n"]} 2020-05-24T13:43:16.220 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:16 +0000 (0:00:05.258) 0:02:34.007 ************ 2020-05-24T13:43:16.252 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-05-24T13:43:16.254 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:16.299 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:16 +0000 (0:00:00.078) 0:02:34.086 ************ 2020-05-24T13:43:17.306 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-05-24T13:43:17.309 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-05-24T13:43:17.350 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:17 +0000 (0:00:01.050) 0:02:35.137 ************ 2020-05-24T13:43:18.303 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-05-24T13:43:18.305 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-05-24T13:43:18.357 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:18 +0000 (0:00:01.007) 0:02:36.144 ************ 2020-05-24T13:43:18.421 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-05-24T13:43:18.423 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:18.473 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:18 +0000 (0:00:00.116) 0:02:36.260 ************ 2020-05-24T13:43:18.819 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-05-24T13:43:18.820 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:43:18.862 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:18 +0000 (0:00:00.388) 0:02:36.649 ************ 2020-05-24T13:43:19.105 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-05-24T13:43:19.108 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["nagios-plugins", "2.3.3-2.el7.x86_64 from epel"], ["nrpe", "4.0.3-1.el7.x86_64 from epel"], ["nagios-common", "4.4.5-7.el7.x86_64 from epel"], ["nagios-plugins-load", "2.3.3-2.el7.x86_64 from epel"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package nagios-common.x86_64 0:4.4.3-1.el7 will be updated\n---> Package nagios-common.x86_64 0:4.4.5-7.el7 will be an update\n---> Package nagios-plugins.x86_64 0:2.3.1-1.el7 will be updated\n---> Package nagios-plugins.x86_64 0:2.3.3-2.el7 will be an update\n---> Package nagios-plugins-load.x86_64 0:2.3.1-1.el7 will be updated\n---> Package nagios-plugins-load.x86_64 0:2.3.3-2.el7 will be an update\n---> Package nrpe.x86_64 0:3.2.1-8.el7 will be updated\n---> Package nrpe.x86_64 0:4.0.3-1.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n nagios-common x86_64 4.4.5-7.el7 epel 27 k\n nagios-plugins x86_64 2.3.3-2.el7 epel 113 k\n nagios-plugins-load x86_64 2.3.3-2.el7 epel 41 k\n nrpe x86_64 4.0.3-1.el7 epel 264 k\n\nTransaction Summary\n============================================================================== 2020-05-24T13:43:19.110 INFO:teuthology.task.ansible.out:==\nUpgrade 4 Packages\n\nTotal download size: 446 k\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 3.1 MB/s | 446 kB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : nagios-common-4.4.5-7.el7.x86_64 1/8 \n Updating : nagios-plugins-2.3.3-2.el7.x86_64 2/8 \n Updating : nagios-plugins-load-2.3.3-2.el7.x86_64 3/8 \n Updating : nrpe-4.0.3-1.el7.x86_64 4/8 \nwarning: /etc/nagios/nrpe.cfg created as /etc/nagios/nrpe.cfg.rpmnew\n Cleanup : nagios-plugins-load-2.3.1-1.el7.x86_64 5/8 \n Cleanup : nagios-plugins-2.3.1-1.el7.x86_64 6/8 \n Cleanup : nrpe-3.2.1-8.el7.x86_64 7/8 \n Cleanup : nagios-common-4.4.3-1.el7.x86_64 8/8 \n Verifying : nrpe-4.0.3-1.el7.x86_64 1/8 \n Verifying : nagios-plugins-2.3.3-2.el7.x86_64 2/8 \n Verifying : nagios-plugins-load-2.3.3-2.el7.x86_64 3/8 \n Verifying : nagios-common-4.4.5-7.el7.x86_64 4/8 \n Verifying : nrpe-3.2.1-8.el7.x86_64 5/8 \n Verifying : nagios-plugins-2.3.1-1.el7.x86_64 6/8 \n Verifying : nagios-plugins-load-2.3.1-1.el7.x86_64 7/8 \n Verifying : nagios-common-4.4.3-1.el7.x86_64 8/8 \n\nUpdated:\n nagios-common.x86_64 0:4.4.5-7.el7 nagios-plugins.x86_64 0:2.3.3-2.el7 \n nagios-plugins-load.x86_64 0:2.3.3-2.el7 nrpe.x86_64 0:4.0.3-1.el7 \n\nComplete!\n"]} 2020-05-24T13:43:19.147 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:19 +0000 (0:00:00.284) 0:02:36.934 ************ 2020-05-24T13:43:19.202 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:19.236 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:19 +0000 (0:00:00.089) 0:02:37.023 ************ 2020-05-24T13:43:19.278 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-05-24T13:43:19.281 INFO:teuthology.task.ansible.out:fatal: [smithi160.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-05-24T13:43:19.329 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:19 +0000 (0:00:00.092) 0:02:37.116 ************ 2020-05-24T13:43:19.372 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-05-24T13:43:19.374 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:19.428 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:19 +0000 (0:00:00.099) 0:02:37.215 ************ 2020-05-24T13:43:20.241 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-05-24T13:43:20.288 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:20 +0000 (0:00:00.860) 0:02:38.075 ************ 2020-05-24T13:43:20.469 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-05-24T13:43:20.470 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2020-05-24T13:43:20.518 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:20 +0000 (0:00:00.230) 0:02:38.305 ************ 2020-05-24T13:43:21.263 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-05-24T13:43:21.269 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-05-24T13:43:21.308 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:21 +0000 (0:00:00.789) 0:02:39.095 ************ 2020-05-24T13:43:21.357 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-05-24T13:43:21.359 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-05-24T13:43:21.392 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:21 +0000 (0:00:00.084) 0:02:39.179 ************ 2020-05-24T13:43:21.548 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-05-24T13:43:21.549 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2020-05-24 13:43:14 UTC", "ActiveEnterTimestampMonotonic": "187733469", "ActiveExitTimestamp": "Sun 2020-05-24 13:43:14 UTC", "ActiveExitTimestampMonotonic": "187723062", "ActiveState": "active", "After": "system.slice local-fs.target nss-lookup.target systemd-journald.socket basic.target network.target var-run.mount time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2020-05-24 13:43:14 UTC", "AssertTimestampMonotonic": "187732889", "Before": "getty@tty1.service shutdown.target multi-user.target xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2020-05-24 13:43:14 UTC", "ConditionTimestampMonotonic": "187732889", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "31708", "ExecMainStartTimestamp": "Sun 2020-05-24 13:43:14 UTC", "ExecMainStartTimestampMonotonic": "187733410", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/ 2020-05-24T13:43:21.558 INFO:teuthology.task.ansible.out:usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Sun 2020-05-24 13:43:14 UTC] ; stop_time=[n/a] ; pid=31708 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[Sun 2020-05-24 13:43:14 UTC] ; stop_time=[Sun 2020-05-24 13:43:14 UTC] ; pid=31705 ; code=exited ; status=0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2020-05-24 13:43:14 UTC", "InactiveEnterTimestampMonotonic": "187732188", "InactiveExitTimestamp": "Sun 2020-05-24 13:43:14 UTC", "InactiveExitTimestampMonotonic": "187733469", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "31708", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", 2020-05-24T13:43:21.560 INFO:teuthology.task.ansible.out: "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target network-online.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2020-05-24 13:43:14 UTC", "WatchdogTimestampMonotonic": "187733448", "WatchdogUSec": "0"}} 2020-05-24T13:43:21.591 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:21 +0000 (0:00:00.199) 0:02:39.378 ************ 2020-05-24T13:43:21.750 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-05-24T13:43:21.754 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:43:21.795 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:21 +0000 (0:00:00.203) 0:02:39.582 ************ 2020-05-24T13:43:21.944 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004500", "end": "2020-05-24 13:43:21.903650", "rc": 0, "start": "2020-05-24 13:43:21.899150", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-05-24T13:43:21.979 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:21 +0000 (0:00:00.184) 0:02:39.766 ************ 2020-05-24T13:43:22.192 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-05-24T13:43:22.195 INFO:teuthology.task.ansible.out:fatal: [smithi050.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-05-24T13:43:22.226 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:22 +0000 (0:00:00.246) 0:02:40.013 ************ 2020-05-24T13:43:22.281 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:43:22.315 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:22 +0000 (0:00:00.088) 0:02:40.102 ************ 2020-05-24T13:43:22.526 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2020-05-24T13:43:22.559 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:22 +0000 (0:00:00.244) 0:02:40.346 ************ 2020-05-24T13:43:23.323 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-05-24T13:43:23.325 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2020-05-24T13:43:23.372 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:23 +0000 (0:00:00.812) 0:02:41.159 ************ 2020-05-24T13:43:23.831 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-05-24T13:43:23.832 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2020-05-24 13:43:17 UTC", "ActiveEnterTimestampMonotonic": "208145693", "ActiveExitTimestamp": "Sun 2020-05-24 13:43:17 UTC", "ActiveExitTimestampMonotonic": "208133683", "ActiveState": "active", "After": "system.slice var-run.mount systemd-journald.socket local-fs.target network.target basic.target nss-lookup.target time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2020-05-24 13:43:17 UTC", "AssertTimestampMonotonic": "208144955", "Before": "getty@tty1.service xdm.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2020-05-24 13:43:17 UTC", "ConditionTimestampMonotonic": "208144955", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "31455", "ExecMainStartTimestamp": "Sun 2020-05-24 13:43:17 UTC", "ExecMainStartTimestampMonotonic": "208145636", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/ 2020-05-24T13:43:23.834 INFO:teuthology.task.ansible.out:usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Sun 2020-05-24 13:43:17 UTC] ; stop_time=[n/a] ; pid=31455 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[Sun 2020-05-24 13:43:17 UTC] ; stop_time=[Sun 2020-05-24 13:43:17 UTC] ; pid=31452 ; code=exited ; status=0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2020-05-24 13:43:17 UTC", "InactiveEnterTimestampMonotonic": "208144322", "InactiveExitTimestamp": "Sun 2020-05-24 13:43:17 UTC", "InactiveExitTimestampMonotonic": "208145693", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "31455", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", 2020-05-24T13:43:23.836 INFO:teuthology.task.ansible.out: "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2020-05-24 13:43:17 UTC", "WatchdogTimestampMonotonic": "208145671", "WatchdogUSec": "0"}} 2020-05-24T13:43:23.872 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:23 +0000 (0:00:00.500) 0:02:41.659 ************ 2020-05-24T13:43:24.178 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004055", "end": "2020-05-24 13:43:24.150787", "rc": 0, "start": "2020-05-24 13:43:24.146732", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-05-24T13:43:24.211 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:24 +0000 (0:00:00.338) 0:02:41.998 ************ 2020-05-24T13:43:24.213 INFO:teuthology.task.ansible.out: 2020-05-24T13:43:24.729 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-05-24T13:43:24.731 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2020-05-24T13:43:24.784 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:24 +0000 (0:00:00.573) 0:02:42.571 ************ 2020-05-24T13:43:46.040 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-266.el7.noarch from base"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-266.el7 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-266.el7 for package: selinux-policy-targeted-3.13.1-266.el7.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-266.el7 for package: selinux-policy-targeted-3.13.1-266.el7.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-266.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository\n Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-266.el7 base 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-266.el7 base 497 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal downlo 2020-05-24T13:43:46.043 INFO:teuthology.task.ansible.out:ad size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 16 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-266.el7.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-266.el7.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7_7.6.noarch 4/4 \n Verifying : selinux-policy-targeted-3.13.1-266.el7.noarch 1/4 \n Verifying : selinux-policy-3.13.1-266.el7.noarch 2/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 3/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-266.el7 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-266.el7 \n\nComplete!\n"]} 2020-05-24T13:43:46.082 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:46 +0000 (0:00:21.297) 0:03:03.869 ************ 2020-05-24T13:43:46.795 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-05-24T13:43:46.797 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-05-24T13:43:46.832 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:46 +0000 (0:00:00.748) 0:03:04.617 ************ 2020-05-24T13:43:49.441 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-05-24T13:43:49.443 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-266.el7.noarch from base"]]}, "msg": "", "obsoletes": {"iwl7265-firmware": {"dist": "noarch", "repo": "@anaconda", "version": "22.0.7.0-69.el7"}}, "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-266.el7 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-266.el7 for package: selinux-policy-targeted-3.13.1-266.el7.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-266.el7 for package: selinux-policy-targeted-3.13.1-266.el7.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-266.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository\n Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-266.el7 base 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-266.el7 base 497 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydel 2020-05-24T13:43:49.445 INFO:teuthology.task.ansible.out:tarpm not installed.\n--------------------------------------------------------------------------------\nTotal 16 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-266.el7.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-266.el7.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7_7.6.noarch 4/4 \n Verifying : selinux-policy-targeted-3.13.1-266.el7.noarch 1/4 \n Verifying : selinux-policy-3.13.1-266.el7.noarch 2/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 3/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-266.el7 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-266.el7 \n\nComplete!\n"]} 2020-05-24T13:43:49.477 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:49 +0000 (0:00:02.647) 0:03:07.264 ************ 2020-05-24T13:43:50.031 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-05-24T13:43:50.033 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-05-24T13:43:50.078 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:50 +0000 (0:00:00.600) 0:03:07.865 ************ 2020-05-24T13:43:54.861 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-05-24T13:43:54.863 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.706885", "end": "2020-05-24 13:43:54.820999", "failed_when_result": false, "rc": 0, "start": "2020-05-24 13:43:47.114114", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-05-24T13:43:54.914 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:54 +0000 (0:00:04.836) 0:03:12.701 ************ 2020-05-24T13:43:55.926 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-05-24T13:43:55.928 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-05-24T13:43:55.974 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:55 +0000 (0:00:01.055) 0:03:13.757 ************ 2020-05-24T13:43:56.339 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-05-24T13:43:56.341 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.015736", "end": "2020-05-24 13:43:56.304246", "rc": 0, "start": "2020-05-24 13:43:56.288510", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2020-05-24T13:43:56.373 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:56 +0000 (0:00:00.402) 0:03:14.160 ************ 2020-05-24T13:43:56.733 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-05-24T13:43:56.735 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.011249", "end": "2020-05-24 13:43:56.705439", "rc": 0, "start": "2020-05-24 13:43:56.694190", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:43:56.769 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:56 +0000 (0:00:00.395) 0:03:14.556 ************ 2020-05-24T13:43:58.717 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.286574", "end": "2020-05-24 13:43:58.672383", "failed_when_result": false, "rc": 0, "start": "2020-05-24 13:43:50.385809", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-05-24T13:43:58.764 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:58 +0000 (0:00:01.994) 0:03:16.551 ************ 2020-05-24T13:43:59.804 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-05-24T13:43:59.806 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-05-24T13:43:59.844 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:43:59 +0000 (0:00:01.079) 0:03:17.631 ************ 2020-05-24T13:44:00.193 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-05-24T13:44:00.195 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.017373", "end": "2020-05-24 13:44:00.157055", "rc": 0, "start": "2020-05-24 13:44:00.139682", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2020-05-24T13:44:00.233 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:00 +0000 (0:00:00.386) 0:03:18.017 ************ 2020-05-24T13:44:00.561 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-05-24T13:44:00.563 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.017820", "end": "2020-05-24 13:44:00.530454", "rc": 0, "start": "2020-05-24 13:44:00.512634", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:00.607 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:00 +0000 (0:00:00.377) 0:03:18.394 ************ 2020-05-24T13:44:04.902 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-05-24T13:44:04.903 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.773579", "end": "2020-05-24 13:44:04.858968", "rc": 0, "start": "2020-05-24 13:43:57.085389", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:04.940 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:04 +0000 (0:00:04.332) 0:03:22.727 ************ 2020-05-24T13:44:05.323 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-05-24T13:44:05.325 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-05-24T13:44:05.357 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:05 +0000 (0:00:00.417) 0:03:23.144 ************ 2020-05-24T13:44:05.410 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-05-24T13:44:05.412 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2020-05-24T13:44:05.452 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:05 +0000 (0:00:00.094) 0:03:23.238 ************ 2020-05-24T13:44:05.502 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-05-24T13:44:05.503 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-24T13:44:05.549 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:05 +0000 (0:00:00.094) 0:03:23.333 ************ 2020-05-24T13:44:05.608 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** ok: [smithi160.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "podman-docker", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "python3"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2020-05-24T13:44:05.643 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:05 +0000 (0:00:00.097) 0:03:23.430 ************ 2020-05-24T13:44:05.688 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-05-24T13:44:05.690 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-24T13:44:05.732 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:05 +0000 (0:00:00.085) 0:03:23.516 ************ 2020-05-24T13:44:06.059 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-05-24T13:44:06.061 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-05-24T13:44:06.097 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:06 +0000 (0:00:00.364) 0:03:23.884 ************ 2020-05-24T13:44:06.437 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-05-24T13:44:06.440 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "gid": 1209, "name": "ubuntu", "state": "present", "system": false} 2020-05-24T13:44:06.479 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:06 +0000 (0:00:00.382) 0:03:24.266 ************ 2020-05-24T13:44:06.525 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:06.568 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:06 +0000 (0:00:00.088) 0:03:24.355 ************ 2020-05-24T13:44:06.599 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-05-24T13:44:06.601 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:06.632 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:06 +0000 (0:00:00.063) 0:03:24.419 ************ 2020-05-24T13:44:07.016 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-05-24T13:44:07.018 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-05-24T13:44:07.074 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:07 +0000 (0:00:00.442) 0:03:24.861 ************ 2020-05-24T13:44:08.139 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-05-24T13:44:08.141 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-05-24T13:44:08.174 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:08 +0000 (0:00:01.099) 0:03:25.961 ************ 2020-05-24T13:44:08.226 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-05-24T13:44:08.228 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi160"}, "changed": false} 2020-05-24T13:44:08.259 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:08 +0000 (0:00:00.085) 0:03:26.046 ************ 2020-05-24T13:44:09.225 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-05-24T13:44:09.227 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.338747", "end": "2020-05-24 13:44:09.188211", "rc": 0, "start": "2020-05-24 13:44:00.849464", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:09.263 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:09 +0000 (0:00:01.003) 0:03:27.050 ************ 2020-05-24T13:44:09.334 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-05-24T13:44:09.337 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi160.front.sepia.ceph.com", "ansible_hostname": "smithi160", "ansible_nodename": "smithi160"}, "changed": true, "name": "smithi160"} 2020-05-24T13:44:09.370 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:09 +0000 (0:00:00.106) 0:03:27.157 ************ 2020-05-24T13:44:09.619 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-05-24T13:44:09.622 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-05-24T13:44:09.677 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:09 +0000 (0:00:00.307) 0:03:27.464 ************ 2020-05-24T13:44:09.726 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-05-24T13:44:09.728 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2020-05-24T13:44:09.781 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:09 +0000 (0:00:00.104) 0:03:27.568 ************ 2020-05-24T13:44:09.842 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi050.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-24T13:44:09.876 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:09 +0000 (0:00:00.094) 0:03:27.663 ************ 2020-05-24T13:44:09.945 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** ok: [smithi050.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "podman-docker", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "python3"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2020-05-24T13:44:09.991 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:09 +0000 (0:00:00.115) 0:03:27.778 ************ 2020-05-24T13:44:10.037 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-05-24T13:44:10.038 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-05-24T13:44:10.080 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:10 +0000 (0:00:00.089) 0:03:27.867 ************ 2020-05-24T13:44:10.327 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2020-05-24T13:44:10.366 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:10 +0000 (0:00:00.286) 0:03:28.153 ************ 2020-05-24T13:44:10.440 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-05-24T13:44:10.442 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-05-24T13:44:10.492 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:10 +0000 (0:00:00.125) 0:03:28.279 ************ 2020-05-24T13:44:10.843 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-05-24T13:44:10.845 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "gid": 1209, "name": "ubuntu", "state": "present", "system": false} 2020-05-24T13:44:10.891 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:10 +0000 (0:00:00.399) 0:03:28.678 ************ 2020-05-24T13:44:10.922 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-05-24T13:44:10.924 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:10.977 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:10 +0000 (0:00:00.085) 0:03:28.764 ************ 2020-05-24T13:44:11.008 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-05-24T13:44:11.012 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:11.042 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:11 +0000 (0:00:00.065) 0:03:28.829 ************ 2020-05-24T13:44:11.356 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-05-24T13:44:11.358 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-05-24T13:44:11.391 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:11 +0000 (0:00:00.348) 0:03:29.178 ************ 2020-05-24T13:44:11.414 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-05-24T13:44:11.416 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-05-24T13:44:11.455 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:11 +0000 (0:00:00.064) 0:03:29.242 ************ 2020-05-24T13:44:12.267 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-05-24T13:44:12.274 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-05-24T13:44:12.302 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:12 +0000 (0:00:00.846) 0:03:30.089 ************ 2020-05-24T13:44:12.542 INFO:teuthology.task.ansible.out: 2020-05-24T13:44:12.544 INFO:teuthology.task.ansible.out:TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-05-24T13:44:12.577 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:12 +0000 (0:00:00.275) 0:03:30.364 ************ 2020-05-24T13:44:12.619 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-05-24T13:44:12.621 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi050"}, "changed": false} 2020-05-24T13:44:12.670 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:12 +0000 (0:00:00.088) 0:03:30.452 ************ 2020-05-24T13:44:13.238 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-05-24T13:44:13.240 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi050.front.sepia.ceph.com", "ansible_hostname": "smithi050", "ansible_nodename": "smithi050"}, "changed": true, "name": "smithi050"} 2020-05-24T13:44:13.275 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:13 +0000 (0:00:00.609) 0:03:31.062 ************ 2020-05-24T13:44:13.295 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-05-24T13:44:13.297 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-05-24T13:44:13.345 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:13 +0000 (0:00:00.070) 0:03:31.132 ************ 2020-05-24T13:44:13.682 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-24T13:44:13.684 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-05-24T13:44:13.728 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:13 +0000 (0:00:00.382) 0:03:31.515 ************ 2020-05-24T13:44:14.068 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-05-24T13:44:14.070 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "kvm", "state": "present", "system": false} 2020-05-24T13:44:14.112 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:14 +0000 (0:00:00.384) 0:03:31.899 ************ 2020-05-24T13:44:14.245 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-05-24T13:44:14.247 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2020-05-24T13:44:14.296 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:14 +0000 (0:00:00.184) 0:03:32.083 ************ 2020-05-24T13:44:14.481 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-24T13:44:14.483 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1209, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-05-24T13:44:14.537 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:14 +0000 (0:00:00.239) 0:03:32.323 ************ 2020-05-24T13:44:15.305 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-05-24T13:44:15.307 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-05-24T13:44:15.346 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:15 +0000 (0:00:00.810) 0:03:33.133 ************ 2020-05-24T13:44:15.514 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-05-24T13:44:15.557 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:15 +0000 (0:00:00.210) 0:03:33.344 ************ 2020-05-24T13:44:16.246 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-05-24T13:44:16.248 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-05-24T13:44:16.297 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:16 +0000 (0:00:00.739) 0:03:34.084 ************ 2020-05-24T13:44:16.556 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-05-24T13:44:16.558 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-05-24T13:44:16.599 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:16 +0000 (0:00:00.301) 0:03:34.386 ************ 2020-05-24T13:44:17.296 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-05-24T13:44:17.298 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-05-24T13:44:17.336 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:17 +0000 (0:00:00.736) 0:03:35.123 ************ 2020-05-24T13:44:17.707 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-24T13:44:17.709 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-05-24T13:44:17.760 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:17 +0000 (0:00:00.424) 0:03:35.547 ************ 2020-05-24T13:44:18.078 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "kvm", "state": "present", "system": false} 2020-05-24T13:44:18.118 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:18 +0000 (0:00:00.355) 0:03:35.903 ************ 2020-05-24T13:44:18.502 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-24T13:44:18.504 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1209, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-05-24T13:44:18.547 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:18 +0000 (0:00:00.430) 0:03:36.333 ************ 2020-05-24T13:44:19.066 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2020-05-24T13:44:19.068 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.123608", "end": "2020-05-24 13:44:19.026052", "rc": 0, "start": "2020-05-24 13:44:16.902444", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:19.108 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:19 +0000 (0:00:00.559) 0:03:36.893 ************ 2020-05-24T13:44:19.151 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:19.191 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:19 +0000 (0:00:00.084) 0:03:36.978 ************ 2020-05-24T13:44:19.221 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-05-24T13:44:19.270 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:19 +0000 (0:00:00.077) 0:03:37.055 ************ 2020-05-24T13:44:19.562 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-05-24T13:44:19.597 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:19 +0000 (0:00:00.328) 0:03:37.384 ************ 2020-05-24T13:44:20.328 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-05-24T13:44:20.330 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'key': 'centos7-fcgi-ceph', 'value': {'name': 'CentOS 7 Local fastcgi Repo', 'baseurl': 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2020-05-24T13:44:20.565 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-05-24T13:44:20.567 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-05-24T13:44:20.632 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:20 +0000 (0:00:01.034) 0:03:38.419 ************ 2020-05-24T13:44:21.301 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-05-24T13:44:21.303 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2020-05-24T13:44:21.354 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:21 +0000 (0:00:00.715) 0:03:39.135 ************ 2020-05-24T13:44:21.383 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-05-24T13:44:21.427 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:21 +0000 (0:00:00.074) 0:03:39.209 ************ 2020-05-24T13:44:21.468 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-05-24T13:44:21.470 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:21.511 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:21 +0000 (0:00:00.088) 0:03:39.298 ************ 2020-05-24T13:44:21.563 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-05-24T13:44:21.565 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:44:21.606 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:21 +0000 (0:00:00.095) 0:03:39.393 ************ 2020-05-24T13:44:21.654 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-05-24T13:44:21.657 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:21.696 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:21 +0000 (0:00:00.088) 0:03:39.482 ************ 2020-05-24T13:44:21.745 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:44:21.747 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-05-24T13:44:21.779 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:21 +0000 (0:00:00.083) 0:03:39.566 ************ 2020-05-24T13:44:22.319 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2020-05-24T13:44:22.358 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:22 +0000 (0:00:00.579) 0:03:40.145 ************ 2020-05-24T13:44:22.922 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-05-24T13:44:22.926 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.230223", "end": "2020-05-24 13:44:22.880625", "rc": 0, "start": "2020-05-24 13:44:22.650402", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirrors.mit.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.facebook.net", " * updates: mirror.vcu.edu", "No unfinished transactions left."]} 2020-05-24T13:44:22.960 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:22 +0000 (0:00:00.600) 0:03:40.745 ************ 2020-05-24T13:44:23.198 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-05-24T13:44:23.200 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.232411", "end": "2020-05-24 13:44:23.167169", "rc": 0, "start": "2020-05-24 13:44:20.934758", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:44:23.240 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:23 +0000 (0:00:00.281) 0:03:41.027 ************ 2020-05-24T13:44:23.292 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-05-24T13:44:23.294 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:23.337 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:23 +0000 (0:00:00.097) 0:03:41.124 ************ 2020-05-24T13:44:23.357 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-05-24T13:44:23.359 INFO:teuthology.task.ansible.out:fatal: [smithi160.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.056570", "end": "2020-05-24 13:44:23.318068", "msg": "non-zero return code", "rc": 1, "start": "2020-05-24 13:44:23.261498", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-05-24T13:44:23.392 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:23 +0000 (0:00:00.054) 0:03:41.179 ************ 2020-05-24T13:44:23.404 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-05-24T13:44:23.452 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:23 +0000 (0:00:00.058) 0:03:41.238 ************ 2020-05-24T13:44:23.460 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:23.506 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:23 +0000 (0:00:00.054) 0:03:41.292 ************ 2020-05-24T13:44:24.470 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-05-24T13:44:24.472 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'key': 'centos7-fcgi-ceph', 'value': {'name': 'CentOS 7 Local fastcgi Repo', 'baseurl': 'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2020-05-24T13:44:25.441 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2020-05-24T13:44:25.479 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:25 +0000 (0:00:01.973) 0:03:43.266 ************ 2020-05-24T13:44:25.519 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-05-24T13:44:25.568 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:25 +0000 (0:00:00.084) 0:03:43.351 ************ 2020-05-24T13:44:25.628 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-05-24T13:44:25.630 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:25.666 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:25 +0000 (0:00:00.101) 0:03:43.453 ************ 2020-05-24T13:44:25.738 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-05-24T13:44:25.740 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:44:25.762 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:44:25.817 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:25 +0000 (0:00:00.150) 0:03:43.604 ************ 2020-05-24T13:44:25.866 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-05-24T13:44:25.868 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:25.919 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:25 +0000 (0:00:00.101) 0:03:43.706 ************ 2020-05-24T13:44:25.969 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:44:25.971 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-05-24T13:44:26.005 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:26 +0000 (0:00:00.086) 0:03:43.792 ************ 2020-05-24T13:44:26.534 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2020-05-24T13:44:26.576 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:26 +0000 (0:00:00.570) 0:03:44.363 ************ 2020-05-24T13:44:27.195 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-05-24T13:44:27.197 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.243165", "end": "2020-05-24 13:44:27.142447", "rc": 0, "start": "2020-05-24 13:44:26.899282", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirrors.mit.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.facebook.net", " * updates: mirror.vcu.edu", "No unfinished transactions left."]} 2020-05-24T13:44:27.246 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:27 +0000 (0:00:00.669) 0:03:45.033 ************ 2020-05-24T13:44:27.685 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-05-24T13:44:27.687 INFO:teuthology.task.ansible.out:fatal: [smithi050.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.057173", "end": "2020-05-24 13:44:27.623720", "msg": "non-zero return code", "rc": 1, "start": "2020-05-24 13:44:27.566547", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-05-24T13:44:27.720 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:27 +0000 (0:00:00.474) 0:03:45.507 ************ 2020-05-24T13:44:27.787 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:27.837 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:27 +0000 (0:00:00.117) 0:03:45.624 ************ 2020-05-24T13:44:31.792 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.4.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.4 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.4 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.4 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7. 2020-05-24T13:44:31.794 INFO:teuthology.task.ansible.out:centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.4 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.4 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.4 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 3 Packages (+3 Dependent packages)\n\nInstalled size: 22 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64 1/6 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.4.x86_64 2/6 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64 3/6 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/6 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/6 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 6/6 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/6 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 2/6 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 3/6 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.4.x86_64 4/6 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64 5/6 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64 6/6 \n\nRemoved:\n librados2.x86_64 1:10.2.5-4.el7 librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.4 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.4 \n 2020-05-24T13:44:31.796 INFO:teuthology.task.ansible.out: qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.4 \n\nComplete!\n"]} 2020-05-24T13:44:31.830 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:31 +0000 (0:00:03.988) 0:03:49.613 ************ 2020-05-24T13:44:35.312 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-05-24T13:44:35.314 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2020-05-24T13:44:35.387 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:35 +0000 (0:00:03.561) 0:03:53.174 ************ 2020-05-24T13:44:36.350 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.4.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.4 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.4 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.4 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7. 2020-05-24T13:44:36.352 INFO:teuthology.task.ansible.out:centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.4 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.4 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.4 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 3 Packages (+3 Dependent packages)\n\nInstalled size: 22 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64 1/6 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.4.x86_64 2/6 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64 3/6 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/6 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/6 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 6/6 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/6 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 2/6 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 3/6 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.4.x86_64 4/6 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.4.x86_64 5/6 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.4.x86_64 6/6 \n\nRemoved:\n librados2.x86_64 1:10.2.5-4.el7 librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.4 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.4 \n 2020-05-24T13:44:36.363 INFO:teuthology.task.ansible.out: qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.4 \n\nComplete!\n"]} 2020-05-24T13:44:36.404 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:36 +0000 (0:00:01.016) 0:03:54.191 ************ 2020-05-24T13:44:39.921 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-05-24T13:44:39.922 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2020-05-24T13:44:39.960 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:39 +0000 (0:00:03.555) 0:03:57.747 ************ 2020-05-24T13:44:51.865 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": ["qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-21.el7_7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing va 2020-05-24T13:44:51.867 INFO:teuthology.task.ansible.out:lgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "podman-docker-1.4.4-4.el7.centos.noarch providing podman-docker is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "iozone-3.424-2_ceph.el7.centos.x86_64 providing iozone is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 provi 2020-05-24T13:44:51.869 INFO:teuthology.task.ansible.out:ding java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "python3-3.6.8-10.el7.x86_64 providing python3 is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-173.el7_8.3 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-173.el7_8.3 for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-173.el7_8.3 for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-173.el7_8.3 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-173.el7_8.3 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-28.el7 will be installed\n--> Finished Dependency 2020-05-24T13:44:51.871 INFO:teuthology.task.ansible.out:Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-173.el7_8.3 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-28.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-173.el7_8.3 updates 703 k\n qemu-kvm-common x86_64 10:1.5.3-173.el7_8.3 updates 438 k\n\nTransaction Summary\n================================================================================\nInstall 2 Packages (+5 Dependent packages)\n\nTotal download size: 7.3 M\nInstalled size: 22 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 16 MB/s | 7.3 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-28.el7.x86_64 1/7 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/7 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/7 \n Installing : 10:qemu-kvm-common-1.5.3-173.el7_8.3.x86_64 4/7 \n Installing : 10:qemu-img-1.5.3-173.el7_8.3.x86_64 5/7 \n Installing : 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64 6/7 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \n Verifying : 1:librbd1-10.2.5-4.el7.x86 2020-05-24T13:44:51.872 INFO:teuthology.task.ansible.out:_64 1/7 \n Verifying : boost-random-1.53.0-28.el7.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 2020-05-24T13:44:51.873 INFO:teuthology.task.ansible.out: 4/7 \n Verifying : 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64 5/7 \n Verifying : 10:qemu-kvm-common-1.5.3-173.el7_8.3.x86_64 6/7 \n Verifying : 10:qemu-img-1.5.3-173.el7_8.3.x86_64 7/7 \n\nInstalled:\n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-173.el7_8.3 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-28.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-173.el7_8.3 \n qemu-kvm-common.x86_64 10:1.5.3-173.el7_8.3 \n\nComplete!\n"]} 2020-05-24T13:44:51.934 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:51 +0000 (0:00:11.974) 0:04:09.721 ************ 2020-05-24T13:44:52.763 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-05-24T13:44:52.765 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2020-05-24T13:44:52.797 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:52 +0000 (0:00:00.863) 0:04:10.584 ************ 2020-05-24T13:44:53.329 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-05-24T13:44:53.331 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2020-05-24T13:44:53.384 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:53 +0000 (0:00:00.586) 0:04:11.171 ************ 2020-05-24T13:44:53.448 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:53.481 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:53 +0000 (0:00:00.097) 0:04:11.268 ************ 2020-05-24T13:44:53.958 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-05-24T13:44:53.960 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service system.slice basic.target polkit.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "shutdown.target iptables.service ip6tables.service ebtables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2020-05-24T13:44:53.962 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2020-05-24T13:44:53.964 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "system.slice network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-24T13:44:54.006 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:54 +0000 (0:00:00.524) 0:04:11.792 ************ 2020-05-24T13:44:54.417 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-05-24T13:44:54.419 INFO:teuthology.task.ansible.out:fatal: [smithi160.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-05-24T13:44:54.466 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:54 +0000 (0:00:00.452) 0:04:12.245 ************ 2020-05-24T13:44:55.446 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-05-24T13:44:55.477 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:55 +0000 (0:00:01.018) 0:04:13.264 ************ 2020-05-24T13:44:55.552 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:55.591 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:55 +0000 (0:00:00.113) 0:04:13.378 ************ 2020-05-24T13:44:55.633 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-05-24T13:44:55.674 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:55 +0000 (0:00:00.082) 0:04:13.460 ************ 2020-05-24T13:44:55.732 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-05-24T13:44:55.733 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:55.778 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:55 +0000 (0:00:00.105) 0:04:13.565 ************ 2020-05-24T13:44:55.826 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-05-24T13:44:55.827 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:55.867 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:55 +0000 (0:00:00.086) 0:04:13.652 ************ 2020-05-24T13:44:55.932 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-05-24T13:44:55.934 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:44:55.983 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:55 +0000 (0:00:00.117) 0:04:13.770 ************ 2020-05-24T13:44:56.030 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-05-24T13:44:56.032 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.076 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.093) 0:04:13.863 ************ 2020-05-24T13:44:56.131 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-05-24T13:44:56.177 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.096) 0:04:13.960 ************ 2020-05-24T13:44:56.225 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-05-24T13:44:56.227 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.267 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.093) 0:04:14.053 ************ 2020-05-24T13:44:56.309 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.355 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.088) 0:04:14.142 ************ 2020-05-24T13:44:56.401 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-05-24T13:44:56.403 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.441 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.085) 0:04:14.228 ************ 2020-05-24T13:44:56.489 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-24T13:44:56.492 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.541 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.100) 0:04:14.328 ************ 2020-05-24T13:44:56.604 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-05-24T13:44:56.606 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.659 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.117) 0:04:14.446 ************ 2020-05-24T13:44:56.729 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.787 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.127) 0:04:14.574 ************ 2020-05-24T13:44:56.831 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-05-24T13:44:56.832 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.881 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.089) 0:04:14.663 ************ 2020-05-24T13:44:56.943 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-05-24T13:44:56.946 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:56.982 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:56 +0000 (0:00:00.105) 0:04:14.769 ************ 2020-05-24T13:44:57.040 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-05-24T13:44:57.042 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.078 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.096) 0:04:14.865 ************ 2020-05-24T13:44:57.159 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-05-24T13:44:57.161 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.196 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.117) 0:04:14.983 ************ 2020-05-24T13:44:57.236 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-05-24T13:44:57.245 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.275 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.078) 0:04:15.062 ************ 2020-05-24T13:44:57.321 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.363 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.088) 0:04:15.150 ************ 2020-05-24T13:44:57.395 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-24T13:44:57.396 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": ["qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-21.el7_7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7_7.1.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose 2020-05-24T13:44:57.398 INFO:teuthology.task.ansible.out:is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "podman-docker-1.4.4-4.el7.centos.noarch providing podman-docker is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "iozone-3.424-2_ceph.el7.centos.x86_64 providing iozone is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7_7.1.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch provid 2020-05-24T13:44:57.399 INFO:teuthology.task.ansible.out:ing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "python3-3.6.8-10.el7.x86_64 providing python3 is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirrors.mit.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.facebook.net\n * updates: mirror.vcu.edu\nResolving Dependencies\n--> Running transaction check\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-173.el7_8.3 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-173.el7_8.3 for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-173.el7_8.3 for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-173.el7_8.3 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-173.el7_8.3 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-28.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n============================================= 2020-05-24T13:44:57.401 INFO:teuthology.task.ansible.out:===================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-173.el7_8.3 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-28.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-173.el7_8.3 updates 703 k\n qemu-kvm-common x86_64 10:1.5.3-173.el7_8.3 updates 438 k\n\nTransaction Summary\n================================================================================\nInstall 2 Packages (+5 Dependent packages)\n\nTotal download size: 7.3 M\nInstalled size: 22 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 15 MB/s | 7.3 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-28.el7.x86_64 1/7 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/7 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/7 \n Installing : 10:qemu-kvm-common-1.5.3-173.el7_8.3.x86_64 4/7 \n Installing : 10:qemu-img-1.5.3-173.el7_8.3.x86_64 5/7 \n Installing : 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64 6/7 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : boost-random-1.53.0-28.el7.x 2020-05-24T13:44:57.403 INFO:teuthology.task.ansible.out:86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-173.el7_8.3.x86_64 5/7 \n Verifying : 10:qemu-kvm-common-1.5.3-173.el7_8.3.x86_64 6/7 \n Verifying : 10:qemu-img-1.5.3-173.el7_8.3.x86_64 7/7 \n\nInstalled:\n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-173.el7_8.3 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-28.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-173.el7_8.3 \n qemu-kvm-common.x86_64 10:1.5.3-173.el7_8.3 \n\nComplete!\n"]} 2020-05-24T13:44:57.437 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.067) 0:04:15.217 ************ 2020-05-24T13:44:57.442 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-05-24T13:44:57.444 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.492 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.061) 0:04:15.279 ************ 2020-05-24T13:44:57.536 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-05-24T13:44:57.538 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.587 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.094) 0:04:15.374 ************ 2020-05-24T13:44:57.628 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-24T13:44:57.630 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.687 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.100) 0:04:15.474 ************ 2020-05-24T13:44:57.718 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-05-24T13:44:57.720 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.762 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.074) 0:04:15.548 ************ 2020-05-24T13:44:57.810 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.862 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.100) 0:04:15.649 ************ 2020-05-24T13:44:57.906 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-05-24T13:44:57.908 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:57.963 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:57 +0000 (0:00:00.099) 0:04:15.749 ************ 2020-05-24T13:44:58.016 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.055 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.093) 0:04:15.842 ************ 2020-05-24T13:44:58.084 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.130 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.074) 0:04:15.917 ************ 2020-05-24T13:44:58.182 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.206 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.075) 0:04:15.992 ************ 2020-05-24T13:44:58.244 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.289 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.083) 0:04:16.076 ************ 2020-05-24T13:44:58.299 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-05-24T13:44:58.300 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2020-05-24T13:44:58.349 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.059) 0:04:16.135 ************ 2020-05-24T13:44:58.362 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.406 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.057) 0:04:16.193 ************ 2020-05-24T13:44:58.448 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-05-24T13:44:58.453 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.524 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.111) 0:04:16.304 ************ 2020-05-24T13:44:58.555 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-24T13:44:58.557 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.602 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.085) 0:04:16.389 ************ 2020-05-24T13:44:58.643 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-24T13:44:58.645 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.695 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.092) 0:04:16.482 ************ 2020-05-24T13:44:58.737 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:58.776 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.081) 0:04:16.563 ************ 2020-05-24T13:44:58.911 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2020-05-24T13:44:58.950 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:58 +0000 (0:00:00.171) 0:04:16.735 ************ 2020-05-24T13:44:59.001 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-24T13:44:59.002 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:44:59.052 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:59 +0000 (0:00:00.104) 0:04:16.839 ************ 2020-05-24T13:44:59.141 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:44:59.143 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-05-24T13:44:59.181 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:59 +0000 (0:00:00.128) 0:04:16.967 ************ 2020-05-24T13:44:59.542 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-05-24T13:44:59.544 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.service basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ip6tables.service iptables.service shutdown.target ebtables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2020-05-24T13:44:59.546 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2020-05-24T13:44:59.548 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-24T13:44:59.578 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:59 +0000 (0:00:00.397) 0:04:17.365 ************ 2020-05-24T13:44:59.589 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-24T13:44:59.591 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:44:59.648 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:44:59 +0000 (0:00:00.069) 0:04:17.434 ************ 2020-05-24T13:44:59.992 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi050.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-05-24T13:45:00.049 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.400) 0:04:17.835 ************ 2020-05-24T13:45:00.061 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-24T13:45:00.064 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-05-24T13:45:00.106 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.058) 0:04:17.893 ************ 2020-05-24T13:45:00.154 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-05-24T13:45:00.155 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.235 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.126) 0:04:18.020 ************ 2020-05-24T13:45:00.276 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.344 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.110) 0:04:18.131 ************ 2020-05-24T13:45:00.387 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-05-24T13:45:00.389 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.452 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.107) 0:04:18.238 ************ 2020-05-24T13:45:00.494 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-05-24T13:45:00.496 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-05-24T13:45:00.529 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.077) 0:04:18.316 ************ 2020-05-24T13:45:00.546 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.596 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.066) 0:04:18.382 ************ 2020-05-24T13:45:00.605 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.651 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.054) 0:04:18.437 ************ 2020-05-24T13:45:00.662 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.713 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.062) 0:04:18.500 ************ 2020-05-24T13:45:00.721 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-05-24T13:45:00.765 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.052) 0:04:18.552 ************ 2020-05-24T13:45:00.775 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.820 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.054) 0:04:18.607 ************ 2020-05-24T13:45:00.828 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-05-24T13:45:00.830 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.880 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.059) 0:04:18.667 ************ 2020-05-24T13:45:00.890 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:45:00.892 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:00.935 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:00 +0000 (0:00:00.055) 0:04:18.722 ************ 2020-05-24T13:45:00.955 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-05-24T13:45:00.956 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.005 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.069) 0:04:18.792 ************ 2020-05-24T13:45:01.016 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-05-24T13:45:01.018 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.057 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.052) 0:04:18.844 ************ 2020-05-24T13:45:01.067 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-05-24T13:45:01.069 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.082 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.121 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.064) 0:04:18.908 ************ 2020-05-24T13:45:01.132 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-24T13:45:01.134 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.191 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.070) 0:04:18.978 ************ 2020-05-24T13:45:01.202 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-05-24T13:45:01.204 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.254 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.061) 0:04:19.040 ************ 2020-05-24T13:45:01.266 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.305 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.051) 0:04:19.092 ************ 2020-05-24T13:45:01.334 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-05-24T13:45:01.380 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.075) 0:04:19.167 ************ 2020-05-24T13:45:01.394 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.435 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.054) 0:04:19.222 ************ 2020-05-24T13:45:01.445 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.511 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.076) 0:04:19.298 ************ 2020-05-24T13:45:01.523 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.575 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.063) 0:04:19.362 ************ 2020-05-24T13:45:01.586 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.645 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.070) 0:04:19.432 ************ 2020-05-24T13:45:01.658 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.710 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.059) 0:04:19.492 ************ 2020-05-24T13:45:01.729 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-05-24T13:45:01.736 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.776 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.071) 0:04:19.563 ************ 2020-05-24T13:45:01.786 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-05-24T13:45:01.788 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.827 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.051) 0:04:19.614 ************ 2020-05-24T13:45:01.839 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-24T13:45:01.842 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.883 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.055) 0:04:19.670 ************ 2020-05-24T13:45:01.893 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-05-24T13:45:01.894 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.943 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.059) 0:04:19.730 ************ 2020-05-24T13:45:01.953 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-05-24T13:45:01.955 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:01.998 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:01 +0000 (0:00:00.051) 0:04:19.782 ************ 2020-05-24T13:45:02.005 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-05-24T13:45:02.007 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.042 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.047) 0:04:19.829 ************ 2020-05-24T13:45:02.050 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-24T13:45:02.053 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.096 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.048) 0:04:19.878 ************ 2020-05-24T13:45:02.103 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.153 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.056) 0:04:19.935 ************ 2020-05-24T13:45:02.162 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.217 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.068) 0:04:20.004 ************ 2020-05-24T13:45:02.245 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.299 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.082) 0:04:20.086 ************ 2020-05-24T13:45:02.309 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-05-24T13:45:02.311 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.355 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.055) 0:04:20.141 ************ 2020-05-24T13:45:02.364 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-05-24T13:45:02.365 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.418 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.057) 0:04:20.199 ************ 2020-05-24T13:45:02.421 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-05-24T13:45:02.421 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.461 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.048) 0:04:20.248 ************ 2020-05-24T13:45:02.470 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-05-24T13:45:02.472 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.511 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.050) 0:04:20.298 ************ 2020-05-24T13:45:02.519 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-05-24T13:45:02.522 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.562 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.047) 0:04:20.346 ************ 2020-05-24T13:45:02.570 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.618 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.058) 0:04:20.404 ************ 2020-05-24T13:45:02.634 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.687 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.070) 0:04:20.474 ************ 2020-05-24T13:45:02.699 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-05-24T13:45:02.701 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.750 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.061) 0:04:20.536 ************ 2020-05-24T13:45:02.759 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-05-24T13:45:02.761 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.818 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.068) 0:04:20.605 ************ 2020-05-24T13:45:02.834 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-05-24T13:45:02.877 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.059) 0:04:20.664 ************ 2020-05-24T13:45:02.885 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-05-24T13:45:02.893 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.925 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:02 +0000 (0:00:00.047) 0:04:20.712 ************ 2020-05-24T13:45:02.949 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi160.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:02.961 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-05-24T13:45:02.983 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.028 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.103) 0:04:20.815 ************ 2020-05-24T13:45:03.042 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi160.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.083 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.054) 0:04:20.870 ************ 2020-05-24T13:45:03.093 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-05-24T13:45:03.095 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.146 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.063) 0:04:20.933 ************ 2020-05-24T13:45:03.218 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-05-24T13:45:03.220 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.223 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:03.266 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.115) 0:04:21.048 ************ 2020-05-24T13:45:03.311 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-05-24T13:45:03.315 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.361 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.098) 0:04:21.147 ************ 2020-05-24T13:45:03.363 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:03.417 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.481 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.120) 0:04:21.268 ************ 2020-05-24T13:45:03.483 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:03.494 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi160.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.033678", "end": "2020-05-24 13:45:03.404592", "rc": 0, "start": "2020-05-24 13:45:03.370914", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-05-24T13:45:03.537 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.055) 0:04:21.324 ************ 2020-05-24T13:45:03.548 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-05-24T13:45:03.550 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.621 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.084) 0:04:21.408 ************ 2020-05-24T13:45:03.632 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-05-24T13:45:03.634 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.685 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.063) 0:04:21.472 ************ 2020-05-24T13:45:03.698 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-05-24T13:45:03.701 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.746 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.060) 0:04:21.533 ************ 2020-05-24T13:45:03.756 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-05-24T13:45:03.758 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.817 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.054) 0:04:21.587 ************ 2020-05-24T13:45:03.830 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.877 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.076) 0:04:21.664 ************ 2020-05-24T13:45:03.887 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-05-24T13:45:03.889 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:03.930 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.050) 0:04:21.714 ************ 2020-05-24T13:45:03.941 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-05-24T13:45:03.943 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:04.002 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:03 +0000 (0:00:00.072) 0:04:21.787 ************ 2020-05-24T13:45:04.009 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:04.059 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.059) 0:04:21.846 ************ 2020-05-24T13:45:04.068 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:04.070 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:04.114 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.054) 0:04:21.901 ************ 2020-05-24T13:45:04.154 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-05-24T13:45:04.156 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:04.210 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.094) 0:04:21.995 ************ 2020-05-24T13:45:04.258 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-05-24T13:45:04.260 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:04.298 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.089) 0:04:22.085 ************ 2020-05-24T13:45:04.342 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-05-24T13:45:04.345 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:04.380 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.082) 0:04:22.167 ************ 2020-05-24T13:45:04.438 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-05-24T13:45:04.440 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:04.476 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.094) 0:04:22.262 ************ 2020-05-24T13:45:04.567 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-05-24T13:45:04.569 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2020-05-24T13:45:04.636 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.160) 0:04:22.422 ************ 2020-05-24T13:45:04.726 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi160.front.sepia.ceph.com] => (item={'block_used': 13047413, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277071, 'size_available': 930926882816, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91466, 'block_size': 4096, 'inode_available': 60963510}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277071, "block_size": 4096, "block_total": 240324484, "block_used": 13047413, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963510, "inode_total": 61054976, "inode_used": 91466, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926882816, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}} 2020-05-24T13:45:04.785 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.143) 0:04:22.566 ************ 2020-05-24T13:45:04.842 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-05-24T13:45:04.844 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-05-24T13:45:04.897 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:04 +0000 (0:00:00.117) 0:04:22.684 ************ 2020-05-24T13:45:05.118 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-05-24T13:45:05.120 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011893", "end": "2020-05-24 13:45:05.090470", "rc": 0, "start": "2020-05-24 13:45:05.078577", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:05.153 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:05 +0000 (0:00:00.255) 0:04:22.940 ************ 2020-05-24T13:45:05.207 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-05-24T13:45:05.209 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'block_used': 13047413, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277071, 'size_available': 930926882816, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91466, 'block_size': 4096, 'inode_available': 60963510}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277071, "block_size": 4096, "block_total": 240324484, "block_used": 13047413, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963510, "inode_total": 61054976, "inode_used": 91466, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926882816, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:05.271 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:05 +0000 (0:00:00.117) 0:04:23.057 ************ 2020-05-24T13:45:05.281 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:45:05.334 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:05 +0000 (0:00:00.063) 0:04:23.121 ************ 2020-05-24T13:45:05.356 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-05-24T13:45:05.422 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:05 +0000 (0:00:00.088) 0:04:23.209 ************ 2020-05-24T13:45:05.542 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi160.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XFZB', 'wwn-0x5000c5009292466b'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XFZB-part1', 'wwn-0x5000c5009292466b-part1'], 'uuids': ['4de53983-240a-4436-8d72-98c50b3c74e0']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009292466b', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XFZB', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFZB", "wwn-0x5000c5009292466b"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFZB-part1", "wwn-0x5000c5009292466b-part1"], "labels": [], "masters": [], "uuids": ["4de53983-240a-4436-8d72-98c50b3c74e0"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XFZB", "size": "931.51 GB", "suppor 2020-05-24T13:45:05.545 INFO:teuthology.task.ansible.out:t_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009292466b"}}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:05.712 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-24T13:45:05.714 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-05-24T13:45:05.755 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:05 +0000 (0:00:00.329) 0:04:23.539 ************ 2020-05-24T13:45:05.793 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:05.837 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:05 +0000 (0:00:00.079) 0:04:23.619 ************ 2020-05-24T13:45:05.877 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:05.931 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:05 +0000 (0:00:00.098) 0:04:23.718 ************ 2020-05-24T13:45:05.973 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.025 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.094) 0:04:23.812 ************ 2020-05-24T13:45:06.077 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.129 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.103) 0:04:23.916 ************ 2020-05-24T13:45:06.172 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.213 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.083) 0:04:24.000 ************ 2020-05-24T13:45:06.254 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-24T13:45:06.256 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.313 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.099) 0:04:24.099 ************ 2020-05-24T13:45:06.355 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.400 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.087) 0:04:24.187 ************ 2020-05-24T13:45:06.439 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.483 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.082) 0:04:24.270 ************ 2020-05-24T13:45:06.518 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-05-24T13:45:06.519 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.561 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.070) 0:04:24.341 ************ 2020-05-24T13:45:06.607 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-05-24T13:45:06.609 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.643 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.089) 0:04:24.430 ************ 2020-05-24T13:45:06.735 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-05-24T13:45:06.737 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.778 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.134) 0:04:24.565 ************ 2020-05-24T13:45:06.830 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.859 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.081) 0:04:24.646 ************ 2020-05-24T13:45:06.930 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:06.972 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:06 +0000 (0:00:00.112) 0:04:24.759 ************ 2020-05-24T13:45:07.030 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.060 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.087) 0:04:24.846 ************ 2020-05-24T13:45:07.105 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-05-24T13:45:07.107 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.151 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.091) 0:04:24.938 ************ 2020-05-24T13:45:07.207 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.241 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.090) 0:04:25.028 ************ 2020-05-24T13:45:07.286 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.330 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.088) 0:04:25.117 ************ 2020-05-24T13:45:07.374 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-05-24T13:45:07.384 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.423 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.093) 0:04:25.210 ************ 2020-05-24T13:45:07.433 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-05-24T13:45:07.435 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-RynkBe-v2C0-QDhS-4fr9-05bn-07me-WjrcrQ', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300JY400BGN', 'nvme-nvme.8086-435646543632333330304a5934303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_5', 'vg_nvme-lv_4', 'vg_nvme-lv_3', 'vg_nvme-lv_2', 'vg_nvme-lv_1'], 'partitions': {}, 'model': 'INTEL SSDPEDMD400G4', 'serial': 'CVFT623300JY400BGN', 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.539020", "end": "2020-05-24 13:45:07.347413", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-RynkBe-v2C0-QDhS-4fr9-05bn-07me-WjrcrQ", "nvme-INTEL_SSDPEDMD400G4_CVFT623300JY400BGN", "nvme-nvme.8086-435646543632333330304a5934303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:05.808393", "stderr": "", "stderr_lines": [], "stdo 2020-05-24T13:45:07.438 INFO:teuthology.task.ansible.out:ut": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:07.472 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.516 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.092) 0:04:25.303 ************ 2020-05-24T13:45:07.557 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-05-24T13:45:07.559 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.604 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.087) 0:04:25.391 ************ 2020-05-24T13:45:07.665 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-05-24T13:45:07.667 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.708 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.102) 0:04:25.493 ************ 2020-05-24T13:45:07.741 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.780 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.074) 0:04:25.567 ************ 2020-05-24T13:45:07.812 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-05-24T13:45:07.869 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:07 +0000 (0:00:00.080) 0:04:25.648 ************ 2020-05-24T13:45:07.939 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi050.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:07.961 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-05-24T13:45:07.993 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.007 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.031 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.043 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.054 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.137 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:08 +0000 (0:00:00.275) 0:04:25.923 ************ 2020-05-24T13:45:08.449 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-05-24T13:45:08.451 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004945", "end": "2020-05-24 13:45:08.420392", "rc": 0, "start": "2020-05-24 13:45:08.415447", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-05-24T13:45:08.524 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:08 +0000 (0:00:00.387) 0:04:26.311 ************ 2020-05-24T13:45:08.582 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.623 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:08 +0000 (0:00:00.098) 0:04:26.410 ************ 2020-05-24T13:45:08.678 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.711 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:08 +0000 (0:00:00.086) 0:04:26.496 ************ 2020-05-24T13:45:08.750 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-05-24T13:45:08.752 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.789 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:08 +0000 (0:00:00.080) 0:04:26.576 ************ 2020-05-24T13:45:08.821 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-05-24T13:45:08.822 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWq941famuv07yw78l2DqPK7eQ1iyQF1B6d'], 'uuids': ['683a476a-885a-476d-971d-a58c41f649d8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.095186", "end": "2020-05-24 13:45:08.783634", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWq941famuv07yw78l2DqPK7eQ1iyQF1B6d"], "labels": [], "masters": [], "uuids": ["683a476a-885a-476d-971d-a58c41f649d8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:07.688448", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:08.846 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:08.931 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:08 +0000 (0:00:00.141) 0:04:26.718 ************ 2020-05-24T13:45:09.466 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2020-05-24T13:45:09.498 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:09 +0000 (0:00:00.566) 0:04:27.285 ************ 2020-05-24T13:45:09.564 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-05-24T13:45:09.567 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item={'block_used': 13047426, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277058, 'size_available': 930926829568, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91470, 'block_size': 4096, 'inode_available': 60963506}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277058, "block_size": 4096, "block_total": 240324484, "block_used": 13047426, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963506, "inode_total": 61054976, "inode_used": 91470, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926829568, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}} 2020-05-24T13:45:09.604 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:09 +0000 (0:00:00.105) 0:04:27.391 ************ 2020-05-24T13:45:09.962 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-05-24T13:45:09.964 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.014170", "end": "2020-05-24 13:45:09.927334", "rc": 0, "start": "2020-05-24 13:45:09.913164", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:10.000 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:09 +0000 (0:00:00.396) 0:04:27.787 ************ 2020-05-24T13:45:10.065 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-05-24T13:45:10.068 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'block_used': 13047426, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277058, 'size_available': 930926829568, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91470, 'block_size': 4096, 'inode_available': 60963506}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277058, "block_size": 4096, "block_total": 240324484, "block_used": 13047426, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963506, "inode_total": 61054976, "inode_used": 91470, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926829568, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:10.120 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:10 +0000 (0:00:00.119) 0:04:27.907 ************ 2020-05-24T13:45:10.155 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-05-24T13:45:10.202 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:10 +0000 (0:00:00.082) 0:04:27.989 ************ 2020-05-24T13:45:10.263 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-05-24T13:45:10.264 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWq5aIRxSx3de4UUXfX31wSCkDy3kuOkqal'], 'uuids': ['9f389365-e3fe-40c3-ae90-2e6d4b68ca56']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.091369", "end": "2020-05-24 13:45:10.217126", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWq5aIRxSx3de4UUXfX31wSCkDy3kuOkqal"], "labels": [], "masters": [], "uuids": ["9f389365-e3fe-40c3-ae90-2e6d4b68ca56"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:09.125757", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:10.280 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HNFD', 'wwn-0x5000c500871ad2da'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HNFD-part1', 'wwn-0x5000c500871ad2da-part1'], 'uuids': ['4de53983-240a-4436-8d72-98c50b3c74e0']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871ad2da', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4HNFD', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HNFD", "wwn-0x5000c500871ad2da"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HNFD-part1", "wwn-0x5000c500871ad2da-part1"], "labels": [], "masters": [], "uuids": ["4de53983-240a-4436-8d72-98c50b3c74e0"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HNFD", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871ad2da"}}, "skip 2020-05-24T13:45:10.282 INFO:teuthology.task.ansible.out:_reason": "Conditional result was False"} 2020-05-24T13:45:11.674 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWqZkL6QbTNTvnb5Xd8YO4EfIp2eC7BQtYU'], 'uuids': ['42d90890-1b51-436c-862a-5244241bc87d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.073573", "end": "2020-05-24 13:45:11.629290", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWqZkL6QbTNTvnb5Xd8YO4EfIp2eC7BQtYU"], "labels": [], "masters": [], "uuids": ["42d90890-1b51-436c-862a-5244241bc87d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:10.555717", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:12.065 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-KuJfaJ-pBXP-sG1o-fXPr-I4Yv-061K-rNB24s', 'nvme-INTEL_SSDPED1D480GA_PHMB7513001F480DGN', 'nvme-nvme.8086-50484d42373531333030314634383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_5', 'vg_nvme-lv_4', 'vg_nvme-lv_3', 'vg_nvme-lv_2', 'vg_nvme-lv_1'], 'partitions': {}, 'model': 'INTEL SSDPED1D480GA', 'serial': 'PHMB7513001F480DGN', 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.452036", "end": "2020-05-24 13:45:12.023823", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-KuJfaJ-pBXP-sG1o-fXPr-I4Yv-061K-rNB24s", "nvme-INTEL_SSDPED1D480GA_PHMB7513001F480DGN", "nvme-nvme.8086-50484d42373531333030314634383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:10.571787", "stderr": "", "stderr_lines": [], "stdout": "Creating 2020-05-24T13:45:12.068 INFO:teuthology.task.ansible.out: new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:13.117 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWqDB7DLTmdaqk5Ee2m6MkrPEREctypobF3'], 'uuids': ['85f49469-c1a3-4d48-94b0-9ef7c7a434aa']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.097046", "end": "2020-05-24 13:45:13.063908", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWqDB7DLTmdaqk5Ee2m6MkrPEREctypobF3"], "labels": [], "masters": [], "uuids": ["85f49469-c1a3-4d48-94b0-9ef7c7a434aa"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:11.966862", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:13.454 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRUPVvqd50DwVcfJIjHjMA2i10F0vv7Ee9'], 'uuids': ['493e05c7-eaa0-42a0-a76d-10aeb7863e3f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.051394", "end": "2020-05-24 13:45:13.408910", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRUPVvqd50DwVcfJIjHjMA2i10F0vv7Ee9"], "labels": [], "masters": [], "uuids": ["493e05c7-eaa0-42a0-a76d-10aeb7863e3f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:12.357516", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:14.534 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWql5O7Dk450O8SHeZHQEyQTmCcHFiGKX46'], 'uuids': ['8063e22f-c197-4f1b-b7c4-a798909317ce']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.082165", "end": "2020-05-24 13:45:14.478011", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWql5O7Dk450O8SHeZHQEyQTmCcHFiGKX46"], "labels": [], "masters": [], "uuids": ["8063e22f-c197-4f1b-b7c4-a798909317ce"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:13.395846", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:14.588 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:14 +0000 (0:00:04.385) 0:04:32.375 ************ 2020-05-24T13:45:14.668 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi160.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XFZB', 'wwn-0x5000c5009292466b'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XFZB-part1', 'wwn-0x5000c5009292466b-part1'], 'uuids': ['4de53983-240a-4436-8d72-98c50b3c74e0']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009292466b', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XFZB', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFZB", "wwn-0x5000c5009292466b"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFZB-part1", "wwn-0x5000c5009292466b-part1"], "labels": [], "masters": [], "uuids": ["4de53983-240a-4436-8d72-98c50b3c74e0"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XFZB", "size": "931.51 GB", "suppor 2020-05-24T13:45:14.672 INFO:teuthology.task.ansible.out:t_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009292466b"}}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:14.869 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-05-24T13:45:14.874 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRlIyN8xFb1BoKf2pqWh25a31Yx0JDb8Wr'], 'uuids': ['96923bda-941d-4c20-ba08-feef09b2bba0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.074785", "end": "2020-05-24 13:45:14.818060", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRlIyN8xFb1BoKf2pqWh25a31Yx0JDb8Wr"], "labels": [], "masters": [], "uuids": ["96923bda-941d-4c20-ba08-feef09b2bba0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:13.743275", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:15.045 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:15.047 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-RynkBe-v2C0-QDhS-4fr9-05bn-07me-WjrcrQ', 'nvme-INTEL_SSDPEDMD400G4_CVFT623300JY400BGN', 'nvme-nvme.8086-435646543632333330304a5934303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_5', 'vg_nvme-lv_4', 'vg_nvme-lv_3', 'vg_nvme-lv_2', 'vg_nvme-lv_1'], 'partitions': {}, 'model': 'INTEL SSDPEDMD400G4', 'serial': 'CVFT623300JY400BGN', 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.062800", "end": "2020-05-24 13:45:15.010824", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-RynkBe-v2C0-QDhS-4fr9-05bn-07me-WjrcrQ", "nvme-INTEL_SSDPEDMD400G4_CVFT623300JY400BGN", "nvme-nvme.8086-435646543632333330304a5934303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:14.948024", "stderr": "110+0 records in\n110+0 records out\n115343 2020-05-24T13:45:15.050 INFO:teuthology.task.ansible.out:360 bytes (115 MB) copied, 0.0567809 s, 2.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0567809 s, 2.0 GB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:15.533 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWq941famuv07yw78l2DqPK7eQ1iyQF1B6d'], 'uuids': ['683a476a-885a-476d-971d-a58c41f649d8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.211035", "end": "2020-05-24 13:45:15.496938", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWq941famuv07yw78l2DqPK7eQ1iyQF1B6d"], "labels": [], "masters": [], "uuids": ["683a476a-885a-476d-971d-a58c41f649d8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:15.285903", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.206067 s, 560 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.206067 s, 560 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:16.064 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWq5aIRxSx3de4UUXfX31wSCkDy3kuOkqal'], 'uuids': ['9f389365-e3fe-40c3-ae90-2e6d4b68ca56']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.202409", "end": "2020-05-24 13:45:16.024689", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWq5aIRxSx3de4UUXfX31wSCkDy3kuOkqal"], "labels": [], "masters": [], "uuids": ["9f389365-e3fe-40c3-ae90-2e6d4b68ca56"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:15.822280", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.195436 s, 590 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.195436 s, 590 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:16.271 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-05-24T13:45:16.278 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRg8rirowHwKV9hWkUv5XcPs0q5fOxA7IK'], 'uuids': ['023b181b-4d8e-46ed-a976-034e526558f8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.063136", "end": "2020-05-24 13:45:16.235896", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRg8rirowHwKV9hWkUv5XcPs0q5fOxA7IK"], "labels": [], "masters": [], "uuids": ["023b181b-4d8e-46ed-a976-034e526558f8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:15.172760", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:16.547 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:16.554 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWqZkL6QbTNTvnb5Xd8YO4EfIp2eC7BQtYU'], 'uuids': ['42d90890-1b51-436c-862a-5244241bc87d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.170555", "end": "2020-05-24 13:45:16.512944", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWqZkL6QbTNTvnb5Xd8YO4EfIp2eC7BQtYU"], "labels": [], "masters": [], "uuids": ["42d90890-1b51-436c-862a-5244241bc87d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:16.342389", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.164173 s, 703 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.164173 s, 703 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:17.029 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWqDB7DLTmdaqk5Ee2m6MkrPEREctypobF3'], 'uuids': ['85f49469-c1a3-4d48-94b0-9ef7c7a434aa']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.173363", "end": "2020-05-24 13:45:16.980515", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWqDB7DLTmdaqk5Ee2m6MkrPEREctypobF3"], "labels": [], "masters": [], "uuids": ["85f49469-c1a3-4d48-94b0-9ef7c7a434aa"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:16.807152", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.166319 s, 694 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.166319 s, 694 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:17.503 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWql5O7Dk450O8SHeZHQEyQTmCcHFiGKX46'], 'uuids': ['8063e22f-c197-4f1b-b7c4-a798909317ce']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'CVFT623300JY400BGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.165525", "end": "2020-05-24 13:45:17.456848", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-jsHqJcrSqOBPO69pWWuLbxDtVlw3XjWql5O7Dk450O8SHeZHQEyQTmCcHFiGKX46"], "labels": [], "masters": [], "uuids": ["8063e22f-c197-4f1b-b7c4-a798909317ce"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300JY400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:17.291323", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.159545 s, 723 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.159545 s, 723 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:17.542 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:17 +0000 (0:00:02.953) 0:04:35.328 ************ 2020-05-24T13:45:17.682 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-05-24T13:45:17.685 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRCurGL47z0Am8VyNn0a0Neo7HZzH2n2jC'], 'uuids': ['f4e0f89a-15ea-4103-b7ab-2f17820e5b02']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.078140", "end": "2020-05-24 13:45:17.643295", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRCurGL47z0Am8VyNn0a0Neo7HZzH2n2jC"], "labels": [], "masters": [], "uuids": ["f4e0f89a-15ea-4103-b7ab-2f17820e5b02"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:16.565155", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-05-24T13:45:17.961 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi160.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.091234", "end": "2020-05-24 13:45:17.924851", "rc": 0, "start": "2020-05-24 13:45:17.833617", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:17.999 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:17 +0000 (0:00:00.457) 0:04:35.786 ************ 2020-05-24T13:45:18.391 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-05-24T13:45:18.393 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.051982", "end": "2020-05-24 13:45:18.355405", "rc": 0, "start": "2020-05-24 13:45:18.303423", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:18.441 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:18 +0000 (0:00:00.442) 0:04:36.228 ************ 2020-05-24T13:45:18.479 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-05-24T13:45:18.525 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:18 +0000 (0:00:00.083) 0:04:36.312 ************ 2020-05-24T13:45:18.570 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-05-24T13:45:18.572 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:18.611 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:18 +0000 (0:00:00.081) 0:04:36.394 ************ 2020-05-24T13:45:18.636 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-05-24T13:45:18.638 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:18.673 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:18 +0000 (0:00:00.063) 0:04:36.458 ************ 2020-05-24T13:45:18.712 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-24T13:45:18.714 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:18.769 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:18 +0000 (0:00:00.098) 0:04:36.556 ************ 2020-05-24T13:45:18.814 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:18.850 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:18 +0000 (0:00:00.080) 0:04:36.637 ************ 2020-05-24T13:45:19.074 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRDo2av6Q1L8gLBh0PY7DL1icG2g6Cl8Wh'], 'uuids': ['70d3cde4-6c9a-4968-8ecb-8a6cabdd5941']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.048841", "end": "2020-05-24 13:45:19.028484", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRDo2av6Q1L8gLBh0PY7DL1icG2g6Cl8Wh"], "labels": [], "masters": [], "uuids": ["70d3cde4-6c9a-4968-8ecb-8a6cabdd5941"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:17.979643", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other u 2020-05-24T13:45:19.075 INFO:teuthology.task.ansible.out:tilities."]} 2020-05-24T13:45:19.122 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.272) 0:04:36.909 ************ 2020-05-24T13:45:19.181 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-05-24T13:45:19.183 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:45:19.258 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.135) 0:04:37.045 ************ 2020-05-24T13:45:19.269 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:19.272 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HNFD', 'wwn-0x5000c500871ad2da'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HNFD-part1', 'wwn-0x5000c500871ad2da-part1'], 'uuids': ['4de53983-240a-4436-8d72-98c50b3c74e0']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871ad2da', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4HNFD', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HNFD", "wwn-0x5000c500871ad2da"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HNFD-part1", "wwn-0x5000c500871ad2da-part1"], "labels": [], "masters": [], "uuids": ["4de53983-240a-4436-8d72-98c50b3c74e0"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HNFD", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871ad2da"}}, "skip 2020-05-24T13:45:19.274 INFO:teuthology.task.ansible.out:_reason": "Conditional result was False"} 2020-05-24T13:45:19.336 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi160.front.sepia.ceph.com] => (item={'block_used': 13047413, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277071, 'size_available': 930926882816, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91466, 'block_size': 4096, 'inode_available': 60963510}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277071, "block_size": 4096, "block_total": 240324484, "block_used": 13047413, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963510, "inode_total": 61054976, "inode_used": 91466, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926882816, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:19.397 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.138) 0:04:37.184 ************ 2020-05-24T13:45:19.441 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:19.484 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.086) 0:04:37.271 ************ 2020-05-24T13:45:19.529 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:19.532 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:19.562 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.078) 0:04:37.349 ************ 2020-05-24T13:45:19.626 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:19.628 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:19.692 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.125) 0:04:37.475 ************ 2020-05-24T13:45:19.701 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-KuJfaJ-pBXP-sG1o-fXPr-I4Yv-061K-rNB24s', 'nvme-INTEL_SSDPED1D480GA_PHMB7513001F480DGN', 'nvme-nvme.8086-50484d42373531333030314634383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': ['vg_nvme-lv_5', 'vg_nvme-lv_4', 'vg_nvme-lv_3', 'vg_nvme-lv_2', 'vg_nvme-lv_1'], 'partitions': {}, 'model': 'INTEL SSDPED1D480GA', 'serial': 'PHMB7513001F480DGN', 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.087168", "end": "2020-05-24 13:45:19.602566", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-KuJfaJ-pBXP-sG1o-fXPr-I4Yv-061K-rNB24s", "nvme-INTEL_SSDPED1D480GA_PHMB7513001F480DGN", "nvme-nvme.8086-50484d42373531333030314634383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13 2020-05-24T13:45:19.703 INFO:teuthology.task.ansible.out::45:19.515398", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0804686 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0804686 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:19.734 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-05-24T13:45:19.736 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:19.768 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.080) 0:04:37.555 ************ 2020-05-24T13:45:19.819 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-05-24T13:45:19.826 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:19.854 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.085) 0:04:37.641 ************ 2020-05-24T13:45:19.899 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-05-24T13:45:19.901 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:19.938 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:19 +0000 (0:00:00.083) 0:04:37.725 ************ 2020-05-24T13:45:20.067 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:20.069 INFO:teuthology.task.ansible.out:TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRUPVvqd50DwVcfJIjHjMA2i10F0vv7Ee9'], 'uuids': ['493e05c7-eaa0-42a0-a76d-10aeb7863e3f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.125189", "end": "2020-05-24 13:45:20.029111", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRUPVvqd50DwVcfJIjHjMA2i10F0vv7Ee9"], "labels": [], "masters": [], "uuids": ["493e05c7-eaa0-42a0-a76d-10aeb7863e3f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:19.903922", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.118642 s, 972 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.118642 s, 972 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:20.545 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRlIyN8xFb1BoKf2pqWh25a31Yx0JDb8Wr'], 'uuids': ['96923bda-941d-4c20-ba08-feef09b2bba0']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.153572", "end": "2020-05-24 13:45:20.500952", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRlIyN8xFb1BoKf2pqWh25a31Yx0JDb8Wr"], "labels": [], "masters": [], "uuids": ["96923bda-941d-4c20-ba08-feef09b2bba0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:20.347380", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.147159 s, 784 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.147159 s, 784 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:20.759 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-05-24T13:45:20.761 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-05-24T13:45:20.807 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:20 +0000 (0:00:00.866) 0:04:38.592 ************ 2020-05-24T13:45:21.019 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:21.020 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRg8rirowHwKV9hWkUv5XcPs0q5fOxA7IK'], 'uuids': ['023b181b-4d8e-46ed-a976-034e526558f8']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.129638", "end": "2020-05-24 13:45:20.974758", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRg8rirowHwKV9hWkUv5XcPs0q5fOxA7IK"], "labels": [], "masters": [], "uuids": ["023b181b-4d8e-46ed-a976-034e526558f8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:20.845120", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.123209 s, 936 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.123209 s, 936 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:21.466 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRCurGL47z0Am8VyNn0a0Neo7HZzH2n2jC'], 'uuids': ['f4e0f89a-15ea-4103-b7ab-2f17820e5b02']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.150162", "end": "2020-05-24 13:45:21.435928", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRCurGL47z0Am8VyNn0a0Neo7HZzH2n2jC"], "labels": [], "masters": [], "uuids": ["f4e0f89a-15ea-4103-b7ab-2f17820e5b02"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:21.285766", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.143536 s, 804 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.143536 s, 804 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:21.598 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:21.600 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:21.948 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-05-24T13:45:21.949 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRDo2av6Q1L8gLBh0PY7DL1icG2g6Cl8Wh'], 'uuids': ['70d3cde4-6c9a-4968-8ecb-8a6cabdd5941']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'holders': [], 'partitions': {}, 'model': None, 'serial': 'PHMB7513001F480DGN', 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.156054", "end": "2020-05-24 13:45:21.916493", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-UcQlukcDuGtBHd1b60gA73Aa6QOkA9VRDo2av6Q1L8gLBh0PY7DL1icG2g6Cl8Wh"], "labels": [], "masters": [], "uuids": ["70d3cde4-6c9a-4968-8ecb-8a6cabdd5941"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB7513001F480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-05-24 13:45:21.760439", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.149187 s, 773 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.149187 s, 773 MB/s"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:21.998 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:21 +0000 (0:00:01.193) 0:04:39.785 ************ 2020-05-24T13:45:22.116 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:22.437 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-05-24T13:45:22.630 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.081643", "end": "2020-05-24 13:45:22.398554", "rc": 0, "start": "2020-05-24 13:45:22.316911", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} Sunday 24 May 2020 13:45:22 +0000 (0:00:00.502) 0:04:40.288 ************ TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:22.934 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-05-24T13:45:22.936 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.053684", "end": "2020-05-24 13:45:22.902000", "rc": 0, "start": "2020-05-24 13:45:22.848316", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:22.982 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:22 +0000 (0:00:00.480) 0:04:40.768 ************ 2020-05-24T13:45:23.027 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-05-24T13:45:23.081 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.100) 0:04:40.868 ************ 2020-05-24T13:45:23.091 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:23.093 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:23.117 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.161 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.079) 0:04:40.948 ************ 2020-05-24T13:45:23.201 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.239 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.077) 0:04:41.026 ************ 2020-05-24T13:45:23.295 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-24T13:45:23.297 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.331 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.092) 0:04:41.118 ************ 2020-05-24T13:45:23.375 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-05-24T13:45:23.377 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.422 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.087) 0:04:41.205 ************ 2020-05-24T13:45:23.566 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:23.612 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.192) 0:04:41.398 ************ 2020-05-24T13:45:23.775 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi050.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-05-24T13:45:23.810 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.198) 0:04:41.597 ************ 2020-05-24T13:45:23.862 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-05-24T13:45:23.864 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'block_used': 13047426, 'uuid': '4de53983-240a-4436-8d72-98c50b3c74e0', 'size_total': 984369086464, 'block_total': 240324484, 'mount': '/', 'block_available': 227277058, 'size_available': 930926829568, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,seclabel,relatime,data=ordered', 'device': '/dev/sda1', 'inode_used': 91470, 'block_size': 4096, 'inode_available': 60963506}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227277058, "block_size": 4096, "block_total": 240324484, "block_used": 13047426, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60963506, "inode_total": 61054976, "inode_used": 91470, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930926829568, "size_total": 984369086464, "uuid": "4de53983-240a-4436-8d72-98c50b3c74e0"}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.906 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.096) 0:04:41.693 ************ 2020-05-24T13:45:23.956 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:23.996 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:23 +0000 (0:00:00.083) 0:04:41.777 ************ 2020-05-24T13:45:23.999 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-05-24T13:45:24.010 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-05-24T13:45:24.039 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.047) 0:04:41.825 ************ 2020-05-24T13:45:24.050 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.099 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.060) 0:04:41.886 ************ 2020-05-24T13:45:24.146 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:24.148 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.198 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.099) 0:04:41.985 ************ 2020-05-24T13:45:24.240 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-05-24T13:45:24.242 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.276 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.078) 0:04:42.063 ************ 2020-05-24T13:45:24.332 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.363 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.086) 0:04:42.150 ************ 2020-05-24T13:45:24.403 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-05-24T13:45:24.405 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:24.438 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:24 +0000 (0:00:00.074) 0:04:42.225 ************ 2020-05-24T13:45:24.454 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:24.719 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:25.035 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-05-24T13:45:25.038 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:25.077 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.639) 0:04:42.864 ************ 2020-05-24T13:45:25.345 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:25.361 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:25.411 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.333) 0:04:43.197 ************ 2020-05-24T13:45:25.634 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:25.751 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:45:25.792 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.381) 0:04:43.579 ************ 2020-05-24T13:45:25.855 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-05-24T13:45:25.857 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-05-24T13:45:25.895 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:25 +0000 (0:00:00.100) 0:04:43.680 ************ 2020-05-24T13:45:26.147 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:26.149 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:26.660 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:26.702 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi160.front.sepia.ceph.com] => {"changed": true} 2020-05-24T13:45:26.763 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:26 +0000 (0:00:00.870) 0:04:44.550 ************ 2020-05-24T13:45:27.143 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-05-24T13:45:27.144 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:27.598 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-05-24T13:45:27.666 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:27 +0000 (0:00:00.903) 0:04:45.453 ************ 2020-05-24T13:45:28.089 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-05-24T13:45:28.131 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.464) 0:04:45.918 ************ 2020-05-24T13:45:28.509 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-24T13:45:28.511 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:28.522 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-05-24T13:45:28.559 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.428) 0:04:46.346 ************ 2020-05-24T13:45:28.612 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-05-24T13:45:28.614 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.653 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.093) 0:04:46.440 ************ 2020-05-24T13:45:28.693 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-05-24T13:45:28.696 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.738 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.083) 0:04:46.524 ************ 2020-05-24T13:45:28.778 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-05-24T13:45:28.780 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.821 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.084) 0:04:46.608 ************ 2020-05-24T13:45:28.832 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-05-24T13:45:28.834 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:28.856 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.895 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.073) 0:04:46.681 ************ 2020-05-24T13:45:28.943 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-05-24T13:45:28.944 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:28.977 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:28 +0000 (0:00:00.082) 0:04:46.764 ************ 2020-05-24T13:45:29.023 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-05-24T13:45:29.025 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.059 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.081) 0:04:46.846 ************ 2020-05-24T13:45:29.110 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-05-24T13:45:29.117 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.159 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.100) 0:04:46.946 ************ 2020-05-24T13:45:29.173 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:29.511 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-05-24T13:45:29.531 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.586 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.422) 0:04:47.369 ************ 2020-05-24T13:45:29.670 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-05-24T13:45:29.672 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2020-05-24T13:45:29.724 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.141) 0:04:47.511 ************ 2020-05-24T13:45:29.776 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-05-24T13:45:29.782 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.817 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.092) 0:04:47.604 ************ 2020-05-24T13:45:29.849 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-05-24T13:45:29.851 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:29.895 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.077) 0:04:47.682 ************ 2020-05-24T13:45:29.942 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-05-24T13:45:29.997 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:29 +0000 (0:00:00.099) 0:04:47.782 ************ 2020-05-24T13:45:30.065 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-05-24T13:45:30.070 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-05-24T13:45:30.115 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.120) 0:04:47.902 ************ 2020-05-24T13:45:30.528 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-05-24T13:45:30.530 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true} 2020-05-24T13:45:30.570 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:30 +0000 (0:00:00.454) 0:04:48.357 ************ 2020-05-24T13:45:30.991 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-05-24T13:45:30.998 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2020-05-24T13:45:31.062 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.492) 0:04:48.849 ************ 2020-05-24T13:45:31.106 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-05-24T13:45:31.162 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.100) 0:04:48.949 ************ 2020-05-24T13:45:31.218 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.048) 0:04:48.998 ************ 2020-05-24T13:45:31.220 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-05-24T13:45:31.221 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.272 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.060) 0:04:49.058 ************ 2020-05-24T13:45:31.318 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.368 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.095) 0:04:49.154 ************ 2020-05-24T13:45:31.430 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-05-24T13:45:31.432 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.469 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.102) 0:04:49.256 ************ 2020-05-24T13:45:31.532 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.576 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.106) 0:04:49.363 ************ 2020-05-24T13:45:31.639 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.694 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.117) 0:04:49.481 ************ 2020-05-24T13:45:31.734 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-05-24T13:45:31.736 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.794 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.100) 0:04:49.581 ************ 2020-05-24T13:45:31.806 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-05-24T13:45:31.808 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "tmp.mount basic.target ntpdate.service system.slice sntp.service -.mount systemd-journald.socket syslog.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target shutdown.target chronyd.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2020-05-24T13:45:31.810 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2020-05-24T13:45:31.812 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-24T13:45:31.851 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.056) 0:04:49.638 ************ 2020-05-24T13:45:31.866 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-05-24T13:45:31.868 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:31.914 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.062) 0:04:49.700 ************ 2020-05-24T13:45:31.925 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:31.929 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-05-24T13:45:31.981 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:31 +0000 (0:00:00.067) 0:04:49.767 ************ 2020-05-24T13:45:32.037 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:32.039 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.084 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.103) 0:04:49.871 ************ 2020-05-24T13:45:32.435 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-05-24T13:45:32.437 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2020-05-24T13:45:32.478 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.393) 0:04:50.265 ************ 2020-05-24T13:45:32.513 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-05-24T13:45:32.515 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.551 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.073) 0:04:50.338 ************ 2020-05-24T13:45:32.596 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-05-24T13:45:32.601 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:32.646 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:32 +0000 (0:00:00.094) 0:04:50.433 ************ 2020-05-24T13:45:33.043 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2020-05-24T13:45:33.114 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:33 +0000 (0:00:00.468) 0:04:50.901 ************ 2020-05-24T13:45:33.169 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-05-24T13:45:33.171 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:33.219 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:33 +0000 (0:00:00.104) 0:04:51.006 ************ 2020-05-24T13:45:33.259 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-05-24T13:45:33.261 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:33.292 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:33 +0000 (0:00:00.073) 0:04:51.079 ************ 2020-05-24T13:45:33.650 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-05-24T13:45:33.652 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "gid": 1209, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-05-24T13:45:33.696 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:33 +0000 (0:00:00.403) 0:04:51.483 ************ 2020-05-24T13:45:33.736 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-05-24T13:45:33.739 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2020-05-24T13:45:33.776 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:33 +0000 (0:00:00.079) 0:04:51.563 ************ 2020-05-24T13:45:33.827 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-05-24T13:45:33.829 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:33.869 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:33 +0000 (0:00:00.092) 0:04:51.656 ************ 2020-05-24T13:45:34.360 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sntp.service systemd-journald.socket ntpdate.service -.mount syslog.target system.slice tmp.mount basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target shutdown.target chronyd.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": " 2020-05-24T13:45:34.362 INFO:teuthology.task.ansible.out:0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551 2020-05-24T13:45:34.364 INFO:teuthology.task.ansible.out:615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-05-24T13:45:34.405 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:34 +0000 (0:00:00.536) 0:04:52.192 ************ 2020-05-24T13:45:34.454 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:34.461 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-05-24T13:45:34.487 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:34 +0000 (0:00:00.081) 0:04:52.274 ************ 2020-05-24T13:45:34.541 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:34.575 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:34 +0000 (0:00:00.088) 0:04:52.362 ************ 2020-05-24T13:45:34.752 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-05-24T13:45:34.757 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1209, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-05-24T13:45:34.787 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:34 +0000 (0:00:00.211) 0:04:52.574 ************ 2020-05-24T13:45:35.540 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-05-24T13:45:35.541 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2020-05-24T13:45:35.577 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:35 +0000 (0:00:00.789) 0:04:53.364 ************ 2020-05-24T13:45:35.608 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:35.643 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:35 +0000 (0:00:00.066) 0:04:53.430 ************ 2020-05-24T13:45:35.678 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-05-24T13:45:35.679 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:35.715 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:35 +0000 (0:00:00.071) 0:04:53.502 ************ 2020-05-24T13:45:36.071 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-05-24T13:45:36.073 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "gid": 1209, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-05-24T13:45:36.116 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:36 +0000 (0:00:00.400) 0:04:53.903 ************ 2020-05-24T13:45:36.626 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-05-24T13:45:36.628 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 20.1.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 20.1.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-05-24T13:45:36.680 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:36 +0000 (0:00:00.563) 0:04:54.466 ************ 2020-05-24T13:45:36.729 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:36.759 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:36 +0000 (0:00:00.079) 0:04:54.546 ************ 2020-05-24T13:45:36.810 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-05-24T13:45:36.812 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-05-24T13:45:36.856 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:36 +0000 (0:00:00.097) 0:04:54.643 ************ 2020-05-24T13:45:37.166 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-05-24T13:45:37.169 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1209, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-05-24T13:45:37.212 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:37 +0000 (0:00:00.356) 0:04:54.999 ************ 2020-05-24T13:45:37.230 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi160.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-05-24T13:45:37.262 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:37 +0000 (0:00:00.049) 0:04:55.049 ************ 2020-05-24T13:45:37.328 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-05-24T13:45:37.329 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:37.379 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:37 +0000 (0:00:00.116) 0:04:55.166 ************ 2020-05-24T13:45:37.428 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-05-24T13:45:37.430 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-05-24T13:45:37.470 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:37 +0000 (0:00:00.088) 0:04:55.255 ************ 2020-05-24T13:45:37.510 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-05-24T13:45:37.511 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:37.582 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:37 +0000 (0:00:00.114) 0:04:55.369 ************ 2020-05-24T13:45:37.920 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi160.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-05-24T13:45:37.959 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:37 +0000 (0:00:00.377) 0:04:55.746 ************ 2020-05-24T13:45:38.320 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-05-24T13:45:38.322 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-05-24T13:45:38.359 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.400) 0:04:56.146 ************ 2020-05-24T13:45:38.405 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-05-24T13:45:38.407 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.453 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.093) 0:04:56.240 ************ 2020-05-24T13:45:38.518 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi160.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.534 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi160.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.590 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.130) 0:04:56.370 ************ 2020-05-24T13:45:38.636 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-05-24T13:45:38.637 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.647 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.660 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.662 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.674 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.708 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.124) 0:04:56.495 ************ 2020-05-24T13:45:38.720 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-05-24T13:45:38.721 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 20.1.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 20.1.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-05-24T13:45:38.762 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.054) 0:04:56.549 ************ 2020-05-24T13:45:38.776 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-05-24T13:45:38.777 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.818 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.055) 0:04:56.605 ************ 2020-05-24T13:45:38.830 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.876 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.058) 0:04:56.663 ************ 2020-05-24T13:45:38.894 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:38.924 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.048) 0:04:56.711 ************ 2020-05-24T13:45:38.933 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-05-24T13:45:38.934 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-05-24T13:45:38.970 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:38 +0000 (0:00:00.045) 0:04:56.757 ************ 2020-05-24T13:45:38.985 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-05-24T13:45:38.987 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:39.026 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.055) 0:04:56.813 ************ 2020-05-24T13:45:39.080 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:39.125 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.098) 0:04:56.912 ************ 2020-05-24T13:45:39.176 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-05-24T13:45:39.177 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:39.217 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.092) 0:04:57.004 ************ 2020-05-24T13:45:39.257 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-05-24T13:45:39.259 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:39.309 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.091) 0:04:57.096 ************ 2020-05-24T13:45:39.352 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-05-24T13:45:39.354 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi050.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-05-24T13:45:39.388 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.079) 0:04:57.175 ************ 2020-05-24T13:45:39.427 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.038) 0:04:57.214 ************ 2020-05-24T13:45:39.452 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-05-24T13:45:39.454 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:39.491 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.064) 0:04:57.278 ************ 2020-05-24T13:45:39.500 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-05-24T13:45:39.502 INFO:teuthology.task.ansible.out:skipping: [smithi160.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:39.538 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.047) 0:04:57.325 ************ 2020-05-24T13:45:39.553 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-05-24T13:45:39.555 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-05-24T13:45:39.596 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.057) 0:04:57.382 ************ 2020-05-24T13:45:39.636 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-05-24T13:45:39.638 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:39.685 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:39 +0000 (0:00:00.088) 0:04:57.471 ************ 2020-05-24T13:45:39.915 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-05-24T13:45:39.916 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.012398", "end": "2020-05-24 13:45:39.861754", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-05-24 13:45:39.849356", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:40.032 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi050.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-05-24T13:45:40.074 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:40 +0000 (0:00:00.390) 0:04:57.861 ************ 2020-05-24T13:45:40.231 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi160.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007748", "end": "2020-05-24 13:45:40.187785", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-05-24 13:45:40.180037", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:40.430 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-05-24T13:45:40.432 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-05-24T13:45:40.465 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:40 +0000 (0:00:00.390) 0:04:58.251 ************ 2020-05-24T13:45:40.495 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.560 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:40 +0000 (0:00:00.095) 0:04:58.347 ************ 2020-05-24T13:45:40.576 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi160.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007281", "end": "2020-05-24 13:45:40.491893", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-05-24 13:45:40.484612", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:40.618 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-05-24T13:45:40.620 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.629 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.640 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.642 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.675 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:40 +0000 (0:00:00.115) 0:04:58.462 ************ 2020-05-24T13:45:40.739 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-05-24T13:45:40.741 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.750 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.762 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.763 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi050.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.812 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:40 +0000 (0:00:00.133) 0:04:58.595 ************ 2020-05-24T13:45:40.849 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-05-24T13:45:40.850 INFO:teuthology.task.ansible.out:ok: [smithi160.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008068", "end": "2020-05-24 13:45:40.809889", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-05-24 13:45:40.801821", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:40.852 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:40.901 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:40 +0000 (0:00:00.092) 0:04:58.688 ************ 2020-05-24T13:45:40.950 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:40 +0000 (0:00:00.047) 0:04:58.736 ************ 2020-05-24T13:45:40.969 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-05-24T13:45:40.971 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:41.011 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:41 +0000 (0:00:00.061) 0:04:58.798 ************ 2020-05-24T13:45:41.020 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-05-24T13:45:41.061 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:41 +0000 (0:00:00.049) 0:04:58.847 ************ 2020-05-24T13:45:41.072 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-05-24T13:45:41.074 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:41.127 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:41 +0000 (0:00:00.067) 0:04:58.914 ************ 2020-05-24T13:45:41.182 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-05-24T13:45:41.191 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:41.215 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:41 +0000 (0:00:00.087) 0:04:59.002 ************ 2020-05-24T13:45:41.257 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:41.298 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:41 +0000 (0:00:00.082) 0:04:59.085 ************ 2020-05-24T13:45:41.300 INFO:teuthology.task.ansible.out: 2020-05-24T13:45:41.327 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:41.364 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:41 +0000 (0:00:00.066) 0:04:59.151 ************ 2020-05-24T13:45:41.399 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-05-24T13:45:41.401 INFO:teuthology.task.ansible.out:skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:41.436 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:41 +0000 (0:00:00.071) 0:04:59.223 ************ 2020-05-24T13:45:41.445 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-05-24T13:45:41.451 INFO:teuthology.task.ansible.out:changed: [smithi160.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-05-24T13:45:41.486 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi050.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-05-24T13:45:41.519 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:41 +0000 (0:00:00.082) 0:04:59.306 ************ 2020-05-24T13:45:41.923 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-05-24T13:45:41.925 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.013964", "end": "2020-05-24 13:45:41.865121", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-05-24 13:45:41.851157", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:42.242 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007259", "end": "2020-05-24 13:45:42.204382", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-05-24 13:45:42.197123", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:42.564 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006526", "end": "2020-05-24 13:45:42.518648", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-05-24 13:45:42.512122", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:42.881 INFO:teuthology.task.ansible.out:ok: [smithi050.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008312", "end": "2020-05-24 13:45:42.842837", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-05-24 13:45:42.834525", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-05-24T13:45:42.915 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:42 +0000 (0:00:01.396) 0:05:00.702 ************ 2020-05-24T13:45:42.968 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-05-24T13:45:43.008 INFO:teuthology.task.ansible.out:Sunday 24 May 2020 13:45:43 +0000 (0:00:00.092) 0:05:00.795 ************ 2020-05-24T13:45:43.339 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-05-24T13:45:43.341 INFO:teuthology.task.ansible.out:changed: [smithi050.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-05-24T13:45:43.440 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-05-24T13:45:43.442 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-05-24T13:45:43.445 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi050.front.sepia.ceph.com : ok=120 changed=42 unreachable=0 failed=0 skipped=164 rescued=0 ignored=3 smithi160.front.sepia.ceph.com : ok=120 changed=42 unreachable=0 failed=0 skipped=164 rescued=0 ignored=3 Sunday 24 May 2020 13:45:43 +0000 (0:00:00.437) 0:05:01.232 ************ =============================================================================== 2020-05-24T13:45:43.447 INFO:teuthology.task.ansible.out:users : Create all admin users with sudo access. ----------------------- 42.28s users : Update authorized_keys using the keys repo --------------------- 38.05s common : nrpe - Ensure SELinux policy is up to date -------------------- 21.30s testnode : Install packages -------------------------------------------- 11.97s common : Make sure smartmontools is installed -------------------------- 11.38s users : Remove revoked users ------------------------------------------- 11.30s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 5.26s common : nrpe - Remove SELinux policy package --------------------------- 4.84s testnode : Zap all non-root disks --------------------------------------- 4.39s common : nrpe - Load SELinux policy package ----------------------------- 4.33s testnode : Ensure ceph packages are not present. ------------------------ 3.99s testnode : Ensure ceph dependency packages are not present. ------------- 3.56s users : Update authorized_keys for each user with literal keys ---------- 2.46s Gathering Facts --------------------------------------------------------- 2.38s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.71s testnode : List any leftover Ceph artifacts from previous jobs ---------- 1.40s common : Configure local epel mirrorlists ------------------------------- 1.20s common : nrpe - Copy SELinux type enforcement file ---------------------- 1.08s testnode : Configure /etc/security/limits.conf -------------------------- 1.03s ansible-managed : Add authorized keys for the ansible user. ------------- 1.03s 2020-05-24T13:45:43.854 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com')] 2020-05-24T13:45:43.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:45:43.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:45:43.970 INFO:teuthology.orchestra.run.smithi050:> true 2020-05-24T13:45:44.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-05-24T13:45:44.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-05-24T13:45:44.474 INFO:teuthology.orchestra.run.smithi160:> true 2020-05-24T13:45:44.840 INFO:teuthology.run_tasks:Running task clock... 2020-05-24T13:45:44.854 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-05-24T13:45:44.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2020-05-24T13:45:44.857 INFO:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-05-24T13:45:44.996 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-05-24T13:45:52.078 INFO:teuthology.orchestra.run.smithi050.stdout:ntpd: time slew -0.001204s 2020-05-24T13:45:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit not found. 2020-05-24T13:45:52.198 INFO:teuthology.orchestra.run.smithi050.stdout: remote refid st t when poll reach delay offset jitter 2020-05-24T13:45:52.199 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================== 2020-05-24T13:45:52.199 INFO:teuthology.orchestra.run.smithi050.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:45:52.199 INFO:teuthology.orchestra.run.smithi050.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:45:52.199 INFO:teuthology.orchestra.run.smithi050.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:45:52.199 INFO:teuthology.orchestra.run.smithi050.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:45:52.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2020-05-24T13:45:52.200 INFO:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-05-24T13:45:52.335 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-05-24T13:45:59.422 INFO:teuthology.orchestra.run.smithi160.stdout:ntpd: time slew -0.000850s 2020-05-24T13:45:59.461 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit not found. 2020-05-24T13:45:59.536 INFO:teuthology.orchestra.run.smithi160.stdout: remote refid st t when poll reach delay offset jitter 2020-05-24T13:45:59.536 INFO:teuthology.orchestra.run.smithi160.stdout:============================================================================== 2020-05-24T13:45:59.537 INFO:teuthology.orchestra.run.smithi160.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:45:59.537 INFO:teuthology.orchestra.run.smithi160.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:45:59.537 INFO:teuthology.orchestra.run.smithi160.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:45:59.537 INFO:teuthology.orchestra.run.smithi160.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-05-24T13:45:59.538 INFO:teuthology.run_tasks:Running task install... 2020-05-24T13:45:59.557 DEBUG:teuthology.task.install:project ceph 2020-05-24T13:45:59.558 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078'}} 2020-05-24T13:45:59.559 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078'} 2020-05-24T13:45:59.559 INFO:teuthology.task.install:Using flavor: notcmalloc 2020-05-24T13:45:59.563 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2020-05-24T13:45:59.564 INFO:teuthology.task.install:extra packages: [] 2020-05-24T13:45:59.564 INFO:teuthology.orchestra.run.smithi050:> sudo lsb_release -is 2020-05-24T13:45:59.734 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS 2020-05-24T13:45:59.736 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:45:59.737 INFO:teuthology.orchestra.run.smithi160:> sudo lsb_release -is 2020-05-24T13:45:59.738 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'tag': None, 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'debuginfo': None, 'flavor': 'notcmalloc', 'downgrade_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'exclude_packages': [], 'extras': None, 'wait_for_package': False, 'project': 'ceph', 'packages': {}, 'install_ceph_packages': True, 'repos_only': False} 2020-05-24T13:45:59.739 INFO:teuthology.orchestra.run.smithi050:> sudo lsb_release -is 2020-05-24T13:45:59.881 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS 2020-05-24T13:45:59.882 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:45:59.883 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:45:59.891 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS 2020-05-24T13:45:59.893 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:45:59.894 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'tag': None, 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'debuginfo': None, 'flavor': 'notcmalloc', 'downgrade_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'exclude_packages': [], 'extras': None, 'wait_for_package': False, 'project': 'ceph', 'packages': {}, 'install_ceph_packages': True, 'repos_only': False} 2020-05-24T13:45:59.894 INFO:teuthology.orchestra.run.smithi160:> sudo lsb_release -is 2020-05-24T13:46:00.040 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS 2020-05-24T13:46:00.042 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:46:00.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:46:00.139 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/ 2020-05-24T13:46:00.140 INFO:teuthology.task.install.rpm:Package version is 14.2.9-537.g6fe217b2dd 2020-05-24T13:46:00.286 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/ 2020-05-24T13:46:00.287 INFO:teuthology.task.install.rpm:Package version is 14.2.9-537.g6fe217b2dd 2020-05-24T13:46:00.359 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-05-24T13:46:00.360 INFO:teuthology.orchestra.run.smithi050:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-05-24T13:46:00.472 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/nautilus/6fe217b2dddadf1baae6a015a57f83f4283a6078/centos/7/flavors/notcmalloc/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-05-24T13:46:00.472 INFO:teuthology.orchestra.run.smithi160:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-05-24T13:46:00.483 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2020-05-24T13:46:00.484 INFO:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6fe217b2dddadf1baae6a015a57f83f4283a6078/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-05-24T13:46:00.599 INFO:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-05-24T13:46:00.600 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2020-05-24T13:46:00.600 INFO:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6fe217b2dddadf1baae6a015a57f83f4283a6078/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-05-24T13:46:00.714 INFO:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-05-24T13:46:00.734 INFO:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-05-24T13:46:00.824 INFO:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-05-24T13:46:00.849 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2020-05-24T13:46:00.851 INFO:teuthology.orchestra.run.smithi050:> sudo yum clean all 2020-05-24T13:46:00.892 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2020-05-24T13:46:00.893 INFO:teuthology.orchestra.run.smithi160:> sudo yum clean all 2020-05-24T13:46:01.053 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:01.091 INFO:teuthology.orchestra.run.smithi160.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2020-05-24T13:46:01.091 INFO:teuthology.orchestra.run.smithi160.stdout: : lab-extras updates 2020-05-24T13:46:01.112 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:01.113 INFO:teuthology.orchestra.run.smithi160.stdout:Cleaning up list of fastest mirrors 2020-05-24T13:46:01.121 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw 2020-05-24T13:46:01.157 INFO:teuthology.orchestra.run.smithi050.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2020-05-24T13:46:01.158 INFO:teuthology.orchestra.run.smithi050.stdout: : lab-extras updates 2020-05-24T13:46:01.176 INFO:teuthology.orchestra.run.smithi050.stdout:Cleaning up list of fastest mirrors 2020-05-24T13:46:01.189 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw 2020-05-24T13:46:01.288 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:01.326 INFO:teuthology.orchestra.run.smithi160.stdout:Determining fastest mirrors 2020-05-24T13:46:01.377 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:01.415 INFO:teuthology.orchestra.run.smithi050.stdout:Determining fastest mirrors 2020-05-24T13:46:01.729 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:46:01.730 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:46:01.730 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:46:01.731 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:46:01.974 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:46:01.975 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:46:01.975 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:46:01.976 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:46:08.187 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:46:08.539 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:46:08.540 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:08.540 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-radosgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.554 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: ceph-base = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.665 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: ceph-selinux = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.666 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: librgw2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.668 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: librados2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.669 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.669 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.671 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.672 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libceph-common.so.0()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.672 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:08.673 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-base.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.682 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: ceph-common = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.683 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: librbd1 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.684 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libcephfs2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.685 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-requests for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.687 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.688 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.689 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-selinux.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.692 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2020-05-24T13:46:08.693 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2020-05-24T13:46:08.693 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2020-05-24T13:46:08.694 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package librados2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be an update 2020-05-24T13:46:08.696 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package librgw2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.697 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:08.697 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-common.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.702 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-rgw = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.702 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libradosstriper1 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.703 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-cephfs = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.704 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-ceph-argparse = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.705 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-rados = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.706 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-rbd = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.707 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.708 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.709 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.710 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.711 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:08.711 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2020-05-24T13:46:08.712 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package libcephfs2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.712 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2020-05-24T13:46:08.713 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package librbd1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be an update 2020-05-24T13:46:08.713 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2020-05-24T13:46:08.714 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2020-05-24T13:46:08.715 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2020-05-24T13:46:08.715 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-requests.noarch 0:2.6.0-9.el7_8 will be installed 2020-05-24T13:46:08.716 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-9.el7_8.noarch 2020-05-24T13:46:08.716 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:08.717 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2020-05-24T13:46:08.718 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package libradosstriper1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.718 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2020-05-24T13:46:08.719 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-ceph-argparse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.720 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-cephfs.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.734 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2020-05-24T13:46:08.735 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-rados.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.735 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-rbd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.735 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-rgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:08.735 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2020-05-24T13:46:08.735 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2020-05-24T13:46:09.433 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:46:09.596 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:46:09.680 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:09.681 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:46:09.684 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:09.684 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:09.684 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository 2020-05-24T13:46:09.685 INFO:teuthology.orchestra.run.smithi160.stdout: Size 2020-05-24T13:46:09.685 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:09.685 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:46:09.685 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.8 M 2020-05-24T13:46:09.685 INFO:teuthology.orchestra.run.smithi160.stdout:Installing for dependencies: 2020-05-24T13:46:09.686 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.4 M 2020-05-24T13:46:09.686 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 19 M 2020-05-24T13:46:09.686 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 20 k 2020-05-24T13:46:09.686 INFO:teuthology.orchestra.run.smithi160.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2020-05-24T13:46:09.686 INFO:teuthology.orchestra.run.smithi160.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2020-05-24T13:46:09.687 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 501 k 2020-05-24T13:46:09.687 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2020-05-24T13:46:09.687 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2020-05-24T13:46:09.687 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 363 k 2020-05-24T13:46:09.687 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2020-05-24T13:46:09.688 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.1 M 2020-05-24T13:46:09.688 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2020-05-24T13:46:09.688 INFO:teuthology.orchestra.run.smithi160.stdout: python-ceph-argparse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 35 k 2020-05-24T13:46:09.688 INFO:teuthology.orchestra.run.smithi160.stdout: python-cephfs x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 111 k 2020-05-24T13:46:09.688 INFO:teuthology.orchestra.run.smithi160.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2020-05-24T13:46:09.688 INFO:teuthology.orchestra.run.smithi160.stdout: python-rados x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 198 k 2020-05-24T13:46:09.689 INFO:teuthology.orchestra.run.smithi160.stdout: python-rbd x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 192 k 2020-05-24T13:46:09.689 INFO:teuthology.orchestra.run.smithi160.stdout: python-requests noarch 2.6.0-9.el7_8 updates 94 k 2020-05-24T13:46:09.689 INFO:teuthology.orchestra.run.smithi160.stdout: python-rgw x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 77 k 2020-05-24T13:46:09.689 INFO:teuthology.orchestra.run.smithi160.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2020-05-24T13:46:09.689 INFO:teuthology.orchestra.run.smithi160.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2020-05-24T13:46:09.690 INFO:teuthology.orchestra.run.smithi160.stdout:Updating for dependencies: 2020-05-24T13:46:09.690 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 3.5 M 2020-05-24T13:46:09.690 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.7 M 2020-05-24T13:46:09.690 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:09.690 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:46:09.691 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:09.691 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package (+21 Dependent packages) 2020-05-24T13:46:09.691 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade ( 2 Dependent packages) 2020-05-24T13:46:09.691 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:09.691 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 43 M 2020-05-24T13:46:09.691 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:46:09.692 INFO:teuthology.orchestra.run.smithi160.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-05-24T13:46:09.793 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:46:09.794 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:09.794 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-radosgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.810 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: ceph-base = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.924 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: ceph-selinux = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.925 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: librgw2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.926 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: librados2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.928 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.928 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.930 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.931 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libceph-common.so.0()(64bit) for package: 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.932 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:09.932 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-base.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.942 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: ceph-common = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.943 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: librbd1 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.945 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libcephfs2 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.946 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-requests for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.948 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.949 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.950 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-selinux.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.953 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2020-05-24T13:46:09.954 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2020-05-24T13:46:09.954 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2020-05-24T13:46:09.955 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package librados2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be an update 2020-05-24T13:46:09.957 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package librgw2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.958 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:09.958 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-common.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.963 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-rgw = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.964 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libradosstriper1 = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.965 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-cephfs = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.968 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-ceph-argparse = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.969 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-rados = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.969 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-rbd = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.969 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.970 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.972 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.972 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.974 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:09.974 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2020-05-24T13:46:09.974 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package libcephfs2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.975 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2020-05-24T13:46:09.975 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package librbd1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be an update 2020-05-24T13:46:09.975 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2020-05-24T13:46:09.976 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2020-05-24T13:46:09.977 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2020-05-24T13:46:09.979 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-requests.noarch 0:2.6.0-9.el7_8 will be installed 2020-05-24T13:46:09.981 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-9.el7_8.noarch 2020-05-24T13:46:09.981 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:09.981 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2020-05-24T13:46:09.981 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package libradosstriper1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.982 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2020-05-24T13:46:09.983 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-ceph-argparse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.983 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-cephfs.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.984 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2020-05-24T13:46:09.984 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-rados.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.985 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-rbd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.985 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-rgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:09.986 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2020-05-24T13:46:09.986 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2020-05-24T13:46:10.899 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:46:10.986 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:10.987 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:46:10.990 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:10.991 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:10.991 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository 2020-05-24T13:46:10.991 INFO:teuthology.orchestra.run.smithi050.stdout: Size 2020-05-24T13:46:10.991 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:10.991 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:46:10.992 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.8 M 2020-05-24T13:46:10.992 INFO:teuthology.orchestra.run.smithi050.stdout:Installing for dependencies: 2020-05-24T13:46:10.992 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.4 M 2020-05-24T13:46:10.992 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 19 M 2020-05-24T13:46:10.993 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 20 k 2020-05-24T13:46:10.993 INFO:teuthology.orchestra.run.smithi050.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2020-05-24T13:46:10.993 INFO:teuthology.orchestra.run.smithi050.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2020-05-24T13:46:10.993 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 501 k 2020-05-24T13:46:10.994 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2020-05-24T13:46:10.994 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2020-05-24T13:46:10.994 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 363 k 2020-05-24T13:46:10.994 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2020-05-24T13:46:10.994 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 5.1 M 2020-05-24T13:46:10.995 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2020-05-24T13:46:10.995 INFO:teuthology.orchestra.run.smithi050.stdout: python-ceph-argparse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 35 k 2020-05-24T13:46:10.995 INFO:teuthology.orchestra.run.smithi050.stdout: python-cephfs x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 111 k 2020-05-24T13:46:10.995 INFO:teuthology.orchestra.run.smithi050.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2020-05-24T13:46:10.995 INFO:teuthology.orchestra.run.smithi050.stdout: python-rados x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 198 k 2020-05-24T13:46:10.995 INFO:teuthology.orchestra.run.smithi050.stdout: python-rbd x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 192 k 2020-05-24T13:46:10.996 INFO:teuthology.orchestra.run.smithi050.stdout: python-requests noarch 2.6.0-9.el7_8 updates 94 k 2020-05-24T13:46:10.996 INFO:teuthology.orchestra.run.smithi050.stdout: python-rgw x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 77 k 2020-05-24T13:46:10.996 INFO:teuthology.orchestra.run.smithi050.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2020-05-24T13:46:10.996 INFO:teuthology.orchestra.run.smithi050.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2020-05-24T13:46:10.996 INFO:teuthology.orchestra.run.smithi050.stdout:Updating for dependencies: 2020-05-24T13:46:10.997 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 3.5 M 2020-05-24T13:46:10.997 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.7 M 2020-05-24T13:46:10.997 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:10.997 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:46:10.997 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:10.999 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package (+21 Dependent packages) 2020-05-24T13:46:10.999 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade ( 2 Dependent packages) 2020-05-24T13:46:10.999 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:10.999 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 43 M 2020-05-24T13:46:10.999 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:46:11.000 INFO:teuthology.orchestra.run.smithi050.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-05-24T13:46:11.894 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:46:11.894 INFO:teuthology.orchestra.run.smithi160.stdout:Total 19 MB/s | 43 MB 00:02 2020-05-24T13:46:11.909 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:46:11.967 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:46:12.212 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:46:12.213 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:46:12.910 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/26 2020-05-24T13:46:13.179 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/26 2020-05-24T13:46:13.443 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:python-ceph-argparse-14.2.9-537.g6fe217b2dd.el7.x86_64 3/26 2020-05-24T13:46:13.997 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/26 2020-05-24T13:46:14.423 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 5/26 2020-05-24T13:46:14.716 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:46:14.716 INFO:teuthology.orchestra.run.smithi050.stdout:Total 12 MB/s | 43 MB 00:03 2020-05-24T13:46:14.741 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:46:14.818 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:46:15.022 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:46:15.023 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:46:15.229 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 6/26 2020-05-24T13:46:15.626 INFO:teuthology.orchestra.run.smithi160.stdout: Updating : 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 7/26 2020-05-24T13:46:15.857 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/26 2020-05-24T13:46:16.145 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/26 2020-05-24T13:46:16.394 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:python-ceph-argparse-14.2.9-537.g6fe217b2dd.el7.x86_64 3/26 2020-05-24T13:46:16.436 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:python-rados-14.2.9-537.g6fe217b2dd.el7.x86_64 8/26 2020-05-24T13:46:16.763 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/26 2020-05-24T13:46:16.788 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:librgw2-14.2.9-537.g6fe217b2dd.el7.x86_64 9/26 2020-05-24T13:46:17.194 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 5/26 2020-05-24T13:46:17.324 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 10/26 2020-05-24T13:46:17.677 INFO:teuthology.orchestra.run.smithi160.stdout: Updating : 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 11/26 2020-05-24T13:46:18.015 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 6/26 2020-05-24T13:46:18.015 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:python-rbd-14.2.9-537.g6fe217b2dd.el7.x86_64 12/26 2020-05-24T13:46:18.316 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:python-cephfs-14.2.9-537.g6fe217b2dd.el7.x86_64 13/26 2020-05-24T13:46:18.454 INFO:teuthology.orchestra.run.smithi050.stdout: Updating : 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 7/26 2020-05-24T13:46:18.712 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:python-rgw-14.2.9-537.g6fe217b2dd.el7.x86_64 14/26 2020-05-24T13:46:19.042 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:libradosstriper1-14.2.9-537.g6fe217b2dd.el7.x86_64 15/26 2020-05-24T13:46:19.385 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 16/26 2020-05-24T13:46:19.408 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:python-rados-14.2.9-537.g6fe217b2dd.el7.x86_64 8/26 2020-05-24T13:46:19.665 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 17/26 2020-05-24T13:46:19.761 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:librgw2-14.2.9-537.g6fe217b2dd.el7.x86_64 9/26 2020-05-24T13:46:20.006 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/26 2020-05-24T13:46:20.293 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 10/26 2020-05-24T13:46:20.382 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 19/26 2020-05-24T13:46:20.658 INFO:teuthology.orchestra.run.smithi050.stdout: Updating : 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 11/26 2020-05-24T13:46:20.994 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:python-rbd-14.2.9-537.g6fe217b2dd.el7.x86_64 12/26 2020-05-24T13:46:21.314 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:python-cephfs-14.2.9-537.g6fe217b2dd.el7.x86_64 13/26 2020-05-24T13:46:21.653 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:python-rgw-14.2.9-537.g6fe217b2dd.el7.x86_64 14/26 2020-05-24T13:46:21.947 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:libradosstriper1-14.2.9-537.g6fe217b2dd.el7.x86_64 15/26 2020-05-24T13:46:22.300 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 16/26 2020-05-24T13:46:22.647 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 17/26 2020-05-24T13:46:22.844 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-requests-2.6.0-9.el7_8.noarch 20/26 2020-05-24T13:46:23.014 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/26 2020-05-24T13:46:23.361 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 19/26 2020-05-24T13:46:23.985 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 21/26 2020-05-24T13:46:24.532 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 22/26 2020-05-24T13:46:25.906 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-requests-2.6.0-9.el7_8.noarch 20/26 2020-05-24T13:46:26.989 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 21/26 2020-05-24T13:46:27.566 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 22/26 2020-05-24T13:46:34.109 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-selinux-14.2.9-537.g6fe217b2dd.el7.x86_64 23/26 2020-05-24T13:46:34.585 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 24/26 2020-05-24T13:46:34.858 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 25/26 2020-05-24T13:46:35.177 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 26/26 2020-05-24T13:46:35.315 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 1/26 2020-05-24T13:46:35.433 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:python-rados-14.2.9-537.g6fe217b2dd.el7.x86_64 2/26 2020-05-24T13:46:35.541 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:librgw2-14.2.9-537.g6fe217b2dd.el7.x86_64 3/26 2020-05-24T13:46:35.675 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/26 2020-05-24T13:46:35.759 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:python-cephfs-14.2.9-537.g6fe217b2dd.el7.x86_64 5/26 2020-05-24T13:46:35.859 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:libradosstriper1-14.2.9-537.g6fe217b2dd.el7.x86_64 6/26 2020-05-24T13:46:35.952 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 7/26 2020-05-24T13:46:36.069 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:python-rgw-14.2.9-537.g6fe217b2dd.el7.x86_64 8/26 2020-05-24T13:46:36.161 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 9/26 2020-05-24T13:46:36.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 10/26 2020-05-24T13:46:36.412 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:python-rbd-14.2.9-537.g6fe217b2dd.el7.x86_64 11/26 2020-05-24T13:46:36.521 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 12/26 2020-05-24T13:46:36.629 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 13/26 2020-05-24T13:46:36.780 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 14/26 2020-05-24T13:46:36.897 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 15/26 2020-05-24T13:46:36.998 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 16/26 2020-05-24T13:46:37.131 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 17/26 2020-05-24T13:46:37.248 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:python-ceph-argparse-14.2.9-537.g6fe217b2dd.el7.x86_64 18/26 2020-05-24T13:46:37.357 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 19/26 2020-05-24T13:46:37.499 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 20/26 2020-05-24T13:46:37.561 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-selinux-14.2.9-537.g6fe217b2dd.el7.x86_64 23/26 2020-05-24T13:46:37.608 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 21/26 2020-05-24T13:46:37.684 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-requests-2.6.0-9.el7_8.noarch 22/26 2020-05-24T13:46:37.776 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-selinux-14.2.9-537.g6fe217b2dd.el7.x86_64 23/26 2020-05-24T13:46:37.902 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 24/26 2020-05-24T13:46:37.914 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 25/26 2020-05-24T13:46:38.010 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 24/26 2020-05-24T13:46:38.271 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 25/26 2020-05-24T13:46:38.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 26/26 2020-05-24T13:46:38.284 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:38.284 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:46:38.284 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.285 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:38.285 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Installed: 2020-05-24T13:46:38.285 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.285 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.286 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.286 INFO:teuthology.orchestra.run.smithi160.stdout: leveldb.x86_64 0:1.12.0-11.el7 2020-05-24T13:46:38.286 INFO:teuthology.orchestra.run.smithi160.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2020-05-24T13:46:38.286 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.286 INFO:teuthology.orchestra.run.smithi160.stdout: liboath.x86_64 0:2.6.2-1.el7 2020-05-24T13:46:38.287 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2020-05-24T13:46:38.287 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.287 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2020-05-24T13:46:38.287 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.289 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2020-05-24T13:46:38.290 INFO:teuthology.orchestra.run.smithi160.stdout: python-ceph-argparse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.290 INFO:teuthology.orchestra.run.smithi160.stdout: python-cephfs.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.290 INFO:teuthology.orchestra.run.smithi160.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2020-05-24T13:46:38.290 INFO:teuthology.orchestra.run.smithi160.stdout: python-rados.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.290 INFO:teuthology.orchestra.run.smithi160.stdout: python-rbd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.291 INFO:teuthology.orchestra.run.smithi160.stdout: python-requests.noarch 0:2.6.0-9.el7_8 2020-05-24T13:46:38.291 INFO:teuthology.orchestra.run.smithi160.stdout: python-rgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.291 INFO:teuthology.orchestra.run.smithi160.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2020-05-24T13:46:38.291 INFO:teuthology.orchestra.run.smithi160.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2020-05-24T13:46:38.291 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:38.292 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Updated: 2020-05-24T13:46:38.292 INFO:teuthology.orchestra.run.smithi160.stdout: librados2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.292 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:38.292 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:38.292 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:46:38.413 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-test 2020-05-24T13:46:38.640 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:38.678 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:46:38.679 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:46:38.680 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:46:38.680 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:46:38.680 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:46:39.290 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:46:39.292 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 26/26 2020-05-24T13:46:39.407 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 1/26 2020-05-24T13:46:39.507 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:python-rados-14.2.9-537.g6fe217b2dd.el7.x86_64 2/26 2020-05-24T13:46:39.614 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:46:39.615 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:39.615 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-test.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:39.624 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:librgw2-14.2.9-537.g6fe217b2dd.el7.x86_64 3/26 2020-05-24T13:46:39.638 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:39.666 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: jq for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:39.667 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: socat for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:39.670 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:39.670 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package jq.x86_64 0:1.6-1.el7 will be installed 2020-05-24T13:46:39.671 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.6-1.el7.x86_64 2020-05-24T13:46:39.671 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2020-05-24T13:46:39.672 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2020-05-24T13:46:39.674 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:39.674 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2020-05-24T13:46:39.708 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/26 2020-05-24T13:46:39.826 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:python-cephfs-14.2.9-537.g6fe217b2dd.el7.x86_64 5/26 2020-05-24T13:46:39.943 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:libradosstriper1-14.2.9-537.g6fe217b2dd.el7.x86_64 6/26 2020-05-24T13:46:40.039 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:46:40.068 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-base-14.2.9-537.g6fe217b2dd.el7.x86_64 7/26 2020-05-24T13:46:40.177 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:python-rgw-14.2.9-537.g6fe217b2dd.el7.x86_64 8/26 2020-05-24T13:46:40.261 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:40.261 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:46:40.262 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:40.262 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:40.263 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository 2020-05-24T13:46:40.263 INFO:teuthology.orchestra.run.smithi160.stdout: Size 2020-05-24T13:46:40.263 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:40.263 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:46:40.263 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 34 M 2020-05-24T13:46:40.264 INFO:teuthology.orchestra.run.smithi160.stdout:Installing for dependencies: 2020-05-24T13:46:40.264 INFO:teuthology.orchestra.run.smithi160.stdout: jq x86_64 1.6-1.el7 epel 167 k 2020-05-24T13:46:40.264 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2020-05-24T13:46:40.264 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2020-05-24T13:46:40.264 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2020-05-24T13:46:40.265 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:40.265 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:46:40.265 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:40.265 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package (+4 Dependent packages) 2020-05-24T13:46:40.265 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:40.266 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 35 M 2020-05-24T13:46:40.266 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 121 M 2020-05-24T13:46:40.266 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:46:40.336 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 9/26 2020-05-24T13:46:40.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 10/26 2020-05-24T13:46:40.562 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:python-rbd-14.2.9-537.g6fe217b2dd.el7.x86_64 11/26 2020-05-24T13:46:40.691 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 12/26 2020-05-24T13:46:40.813 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 13/26 2020-05-24T13:46:40.930 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-radosgw-14.2.9-537.g6fe217b2dd.el7.x86_64 14/26 2020-05-24T13:46:41.047 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 15/26 2020-05-24T13:46:41.160 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 16/26 2020-05-24T13:46:41.273 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-common-14.2.9-537.g6fe217b2dd.el7.x86_64 17/26 2020-05-24T13:46:41.374 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:python-ceph-argparse-14.2.9-537.g6fe217b2dd.el7.x86_64 18/26 2020-05-24T13:46:41.474 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 19/26 2020-05-24T13:46:41.575 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 20/26 2020-05-24T13:46:41.692 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 21/26 2020-05-24T13:46:41.776 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-requests-2.6.0-9.el7_8.noarch 22/26 2020-05-24T13:46:41.901 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-selinux-14.2.9-537.g6fe217b2dd.el7.x86_64 23/26 2020-05-24T13:46:42.019 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 24/26 2020-05-24T13:46:42.020 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 25/26 2020-05-24T13:46:42.135 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:46:42.135 INFO:teuthology.orchestra.run.smithi160.stdout:Total 19 MB/s | 35 MB 00:01 2020-05-24T13:46:42.138 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:46:42.162 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:46:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:46:42.211 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:46:42.410 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 26/26 2020-05-24T13:46:42.410 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:42.411 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:46:42.411 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:42.411 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Installed: 2020-05-24T13:46:42.411 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.412 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.412 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.412 INFO:teuthology.orchestra.run.smithi050.stdout: leveldb.x86_64 0:1.12.0-11.el7 2020-05-24T13:46:42.412 INFO:teuthology.orchestra.run.smithi050.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2020-05-24T13:46:42.412 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.413 INFO:teuthology.orchestra.run.smithi050.stdout: liboath.x86_64 0:2.6.2-1.el7 2020-05-24T13:46:42.413 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2020-05-24T13:46:42.413 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.413 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2020-05-24T13:46:42.413 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.414 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2020-05-24T13:46:42.414 INFO:teuthology.orchestra.run.smithi050.stdout: python-ceph-argparse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.414 INFO:teuthology.orchestra.run.smithi050.stdout: python-cephfs.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.414 INFO:teuthology.orchestra.run.smithi050.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2020-05-24T13:46:42.414 INFO:teuthology.orchestra.run.smithi050.stdout: python-rados.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.414 INFO:teuthology.orchestra.run.smithi050.stdout: python-rbd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.415 INFO:teuthology.orchestra.run.smithi050.stdout: python-requests.noarch 0:2.6.0-9.el7_8 2020-05-24T13:46:42.415 INFO:teuthology.orchestra.run.smithi050.stdout: python-rgw.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.415 INFO:teuthology.orchestra.run.smithi050.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2020-05-24T13:46:42.415 INFO:teuthology.orchestra.run.smithi050.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2020-05-24T13:46:42.415 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:42.416 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Updated: 2020-05-24T13:46:42.416 INFO:teuthology.orchestra.run.smithi050.stdout: librados2.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.416 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:42.416 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:42.416 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:46:42.548 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-test 2020-05-24T13:46:42.876 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:42.922 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:46:42.923 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:46:42.924 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:46:42.924 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:46:42.924 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:46:42.927 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2020-05-24T13:46:43.253 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2020-05-24T13:46:43.584 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2020-05-24T13:46:43.589 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:46:43.936 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:46:43.937 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:43.937 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-test.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:43.958 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:43.991 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: jq for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:43.996 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: socat for package: 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:43.997 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:43.997 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package jq.x86_64 0:1.6-1.el7 will be installed 2020-05-24T13:46:43.997 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.6-1.el7.x86_64 2020-05-24T13:46:43.997 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2020-05-24T13:46:43.997 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2020-05-24T13:46:43.999 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:43.999 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2020-05-24T13:46:44.394 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:46:44.618 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:44.619 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:46:44.620 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:44.620 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:44.621 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository 2020-05-24T13:46:44.621 INFO:teuthology.orchestra.run.smithi050.stdout: Size 2020-05-24T13:46:44.621 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:44.622 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:46:44.622 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 34 M 2020-05-24T13:46:44.622 INFO:teuthology.orchestra.run.smithi050.stdout:Installing for dependencies: 2020-05-24T13:46:44.622 INFO:teuthology.orchestra.run.smithi050.stdout: jq x86_64 1.6-1.el7 epel 167 k 2020-05-24T13:46:44.623 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2020-05-24T13:46:44.623 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2020-05-24T13:46:44.623 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2020-05-24T13:46:44.624 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:44.624 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:46:44.624 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:44.625 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package (+4 Dependent packages) 2020-05-24T13:46:44.625 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:44.625 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 35 M 2020-05-24T13:46:44.626 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 121 M 2020-05-24T13:46:44.626 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:46:46.443 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:46:46.444 INFO:teuthology.orchestra.run.smithi050.stdout:Total 19 MB/s | 35 MB 00:01 2020-05-24T13:46:46.446 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:46:46.474 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:46:46.533 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:46:46.534 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:46:47.198 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2020-05-24T13:46:47.289 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : jq-1.6-1.el7.x86_64 4/5 2020-05-24T13:46:47.526 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2020-05-24T13:46:47.578 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 5/5 2020-05-24T13:46:47.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2020-05-24T13:46:47.810 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : jq-1.6-1.el7.x86_64 2/5 2020-05-24T13:46:47.859 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2020-05-24T13:46:47.935 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2020-05-24T13:46:48.044 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 4/5 2020-05-24T13:46:48.529 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2020-05-24T13:46:48.529 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:48.529 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:46:48.530 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:48.530 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:48.530 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Installed: 2020-05-24T13:46:48.530 INFO:teuthology.orchestra.run.smithi160.stdout: jq.x86_64 0:1.6-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2020-05-24T13:46:48.530 INFO:teuthology.orchestra.run.smithi160.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2020-05-24T13:46:48.530 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:48.531 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:46:48.657 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph 2020-05-24T13:46:48.920 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:48.964 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:46:48.964 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:46:48.965 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:46:48.965 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:46:48.966 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:46:49.606 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:46:49.942 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:46:49.943 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:49.943 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:49.945 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: ceph-mgr = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.976 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: ceph-mds = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.977 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: ceph-mon = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.978 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: ceph-osd = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.980 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:49.980 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-mds.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:49.992 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-mgr.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:49.994 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.995 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.996 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.996 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.997 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:49.998 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-mon.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:50.001 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-osd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:50.006 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:50.006 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2020-05-24T13:46:50.007 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2020-05-24T13:46:50.008 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2020-05-24T13:46:50.009 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:50.010 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:50.010 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:50.011 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:50.012 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:50.013 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2020-05-24T13:46:50.013 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2020-05-24T13:46:50.014 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2020-05-24T13:46:50.015 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:50.015 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2020-05-24T13:46:50.015 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2020-05-24T13:46:50.016 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2020-05-24T13:46:50.017 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2020-05-24T13:46:50.018 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2020-05-24T13:46:50.018 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2020-05-24T13:46:50.018 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2020-05-24T13:46:50.019 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2020-05-24T13:46:50.019 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:50.019 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2020-05-24T13:46:50.019 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2020-05-24T13:46:50.020 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2020-05-24T13:46:50.020 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:50.020 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2020-05-24T13:46:50.021 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2020-05-24T13:46:50.021 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:46:50.021 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2020-05-24T13:46:50.381 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:46:50.608 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:50.608 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:46:50.611 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:50.611 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:50.611 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository 2020-05-24T13:46:50.611 INFO:teuthology.orchestra.run.smithi160.stdout: Size 2020-05-24T13:46:50.612 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:50.612 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:46:50.612 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 2.0 k 2020-05-24T13:46:50.612 INFO:teuthology.orchestra.run.smithi160.stdout:Installing for dependencies: 2020-05-24T13:46:50.613 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.9 M 2020-05-24T13:46:50.613 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.7 M 2020-05-24T13:46:50.613 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 4.4 M 2020-05-24T13:46:50.614 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 16 M 2020-05-24T13:46:50.614 INFO:teuthology.orchestra.run.smithi160.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2020-05-24T13:46:50.614 INFO:teuthology.orchestra.run.smithi160.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2020-05-24T13:46:50.614 INFO:teuthology.orchestra.run.smithi160.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2020-05-24T13:46:50.614 INFO:teuthology.orchestra.run.smithi160.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2020-05-24T13:46:50.615 INFO:teuthology.orchestra.run.smithi160.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2020-05-24T13:46:50.615 INFO:teuthology.orchestra.run.smithi160.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2020-05-24T13:46:50.615 INFO:teuthology.orchestra.run.smithi160.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2020-05-24T13:46:50.615 INFO:teuthology.orchestra.run.smithi160.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2020-05-24T13:46:50.615 INFO:teuthology.orchestra.run.smithi160.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2020-05-24T13:46:50.616 INFO:teuthology.orchestra.run.smithi160.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2020-05-24T13:46:50.616 INFO:teuthology.orchestra.run.smithi160.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2020-05-24T13:46:50.616 INFO:teuthology.orchestra.run.smithi160.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2020-05-24T13:46:50.617 INFO:teuthology.orchestra.run.smithi160.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2020-05-24T13:46:50.617 INFO:teuthology.orchestra.run.smithi160.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2020-05-24T13:46:50.617 INFO:teuthology.orchestra.run.smithi160.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2020-05-24T13:46:50.619 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:50.619 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:46:50.620 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:46:50.620 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package (+19 Dependent packages) 2020-05-24T13:46:50.621 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:46:50.622 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 27 M 2020-05-24T13:46:50.623 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 86 M 2020-05-24T13:46:50.623 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:46:51.651 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : jq-1.6-1.el7.x86_64 4/5 2020-05-24T13:46:52.019 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 5/5 2020-05-24T13:46:52.135 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2020-05-24T13:46:52.161 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:46:52.161 INFO:teuthology.orchestra.run.smithi160.stdout:Total 18 MB/s | 27 MB 00:01 2020-05-24T13:46:52.189 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:46:52.232 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:46:52.244 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : jq-1.6-1.el7.x86_64 2/5 2020-05-24T13:46:52.344 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2020-05-24T13:46:52.478 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:46:52.479 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:46:52.489 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-test-14.2.9-537.g6fe217b2dd.el7.x86_64 4/5 2020-05-24T13:46:52.980 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2020-05-24T13:46:52.980 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:52.980 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:46:52.981 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:46:52.981 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:52.981 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Installed: 2020-05-24T13:46:52.981 INFO:teuthology.orchestra.run.smithi050.stdout: jq.x86_64 0:1.6-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2020-05-24T13:46:52.982 INFO:teuthology.orchestra.run.smithi050.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2020-05-24T13:46:52.982 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:52.982 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:46:53.115 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph 2020-05-24T13:46:53.281 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 1/20 2020-05-24T13:46:53.360 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:46:53.410 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:46:53.410 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:46:53.411 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:46:53.411 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:46:53.411 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:46:53.588 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-webob-1.2.3-7.el7.noarch 2/20 2020-05-24T13:46:54.072 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:46:54.418 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:46:54.419 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:54.419 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:54.422 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: ceph-mgr = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.454 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: ceph-mds = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.455 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: ceph-mon = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.457 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: ceph-osd = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.458 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:54.458 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-mds.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:54.471 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-mgr.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:54.473 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.474 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.475 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.477 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.477 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:46:54.478 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-mon.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:54.481 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-osd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:46:54.486 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:54.487 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2020-05-24T13:46:54.488 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2020-05-24T13:46:54.489 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2020-05-24T13:46:54.490 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:54.490 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:54.491 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:54.492 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:54.493 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2020-05-24T13:46:54.494 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2020-05-24T13:46:54.494 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2020-05-24T13:46:54.495 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2020-05-24T13:46:54.496 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:54.496 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2020-05-24T13:46:54.496 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2020-05-24T13:46:54.496 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2020-05-24T13:46:54.498 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2020-05-24T13:46:54.498 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2020-05-24T13:46:54.498 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2020-05-24T13:46:54.499 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2020-05-24T13:46:54.499 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2020-05-24T13:46:54.499 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:54.499 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2020-05-24T13:46:54.499 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2020-05-24T13:46:54.500 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2020-05-24T13:46:54.501 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:54.501 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2020-05-24T13:46:54.501 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2020-05-24T13:46:54.501 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:46:54.501 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2020-05-24T13:46:54.593 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/20 2020-05-24T13:46:54.885 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:46:55.117 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:55.118 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:46:55.120 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:55.121 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:55.121 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository 2020-05-24T13:46:55.121 INFO:teuthology.orchestra.run.smithi050.stdout: Size 2020-05-24T13:46:55.121 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:55.121 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:46:55.122 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 2.0 k 2020-05-24T13:46:55.122 INFO:teuthology.orchestra.run.smithi050.stdout:Installing for dependencies: 2020-05-24T13:46:55.122 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.9 M 2020-05-24T13:46:55.122 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 1.7 M 2020-05-24T13:46:55.122 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 4.4 M 2020-05-24T13:46:55.123 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 16 M 2020-05-24T13:46:55.123 INFO:teuthology.orchestra.run.smithi050.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2020-05-24T13:46:55.123 INFO:teuthology.orchestra.run.smithi050.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2020-05-24T13:46:55.123 INFO:teuthology.orchestra.run.smithi050.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2020-05-24T13:46:55.123 INFO:teuthology.orchestra.run.smithi050.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2020-05-24T13:46:55.124 INFO:teuthology.orchestra.run.smithi050.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2020-05-24T13:46:55.124 INFO:teuthology.orchestra.run.smithi050.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2020-05-24T13:46:55.124 INFO:teuthology.orchestra.run.smithi050.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2020-05-24T13:46:55.124 INFO:teuthology.orchestra.run.smithi050.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2020-05-24T13:46:55.124 INFO:teuthology.orchestra.run.smithi050.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2020-05-24T13:46:55.124 INFO:teuthology.orchestra.run.smithi050.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2020-05-24T13:46:55.125 INFO:teuthology.orchestra.run.smithi050.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2020-05-24T13:46:55.125 INFO:teuthology.orchestra.run.smithi050.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2020-05-24T13:46:55.125 INFO:teuthology.orchestra.run.smithi050.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2020-05-24T13:46:55.125 INFO:teuthology.orchestra.run.smithi050.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2020-05-24T13:46:55.125 INFO:teuthology.orchestra.run.smithi050.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2020-05-24T13:46:55.126 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:55.126 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:46:55.126 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:46:55.126 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package (+19 Dependent packages) 2020-05-24T13:46:55.126 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:46:55.127 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 27 M 2020-05-24T13:46:55.127 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 86 M 2020-05-24T13:46:55.128 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:46:56.433 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 4/20 2020-05-24T13:46:56.700 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:46:56.700 INFO:teuthology.orchestra.run.smithi050.stdout:Total 17 MB/s | 27 MB 00:01 2020-05-24T13:46:56.725 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:46:56.774 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:46:57.044 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:46:57.045 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:46:57.088 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-osd-14.2.9-537.g6fe217b2dd.el7.x86_64 5/20 2020-05-24T13:46:57.485 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-mds-14.2.9-537.g6fe217b2dd.el7.x86_64 6/20 2020-05-24T13:46:57.795 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 1/20 2020-05-24T13:46:58.038 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 7/20 2020-05-24T13:46:58.198 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-webob-1.2.3-7.el7.noarch 2/20 2020-05-24T13:46:58.369 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 8/20 2020-05-24T13:46:58.509 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/20 2020-05-24T13:46:58.781 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 9/20 2020-05-24T13:46:59.451 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-mako-0.8.1-2.el7.noarch 10/20 2020-05-24T13:47:00.323 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 11/20 2020-05-24T13:47:00.725 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-mon-14.2.9-537.g6fe217b2dd.el7.x86_64 12/20 2020-05-24T13:47:00.928 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-six-1.9.0-0.el7.noarch 13/20 2020-05-24T13:47:01.219 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 4/20 2020-05-24T13:47:01.345 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 14/20 2020-05-24T13:47:01.711 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 15/20 2020-05-24T13:47:01.938 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-osd-14.2.9-537.g6fe217b2dd.el7.x86_64 5/20 2020-05-24T13:47:01.993 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 16/20 2020-05-24T13:47:02.338 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-mds-14.2.9-537.g6fe217b2dd.el7.x86_64 6/20 2020-05-24T13:47:02.454 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 17/20 2020-05-24T13:47:02.865 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 7/20 2020-05-24T13:47:03.185 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 18/20 2020-05-24T13:47:03.265 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 8/20 2020-05-24T13:47:03.626 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 19/20 2020-05-24T13:47:03.663 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 9/20 2020-05-24T13:47:03.838 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 20/20 2020-05-24T13:47:04.131 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-mako-0.8.1-2.el7.noarch 10/20 2020-05-24T13:47:04.132 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 1/20 2020-05-24T13:47:04.282 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 2/20 2020-05-24T13:47:04.392 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 3/20 2020-05-24T13:47:04.509 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 4/20 2020-05-24T13:47:04.645 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 5/20 2020-05-24T13:47:04.768 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 6/20 2020-05-24T13:47:04.877 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 7/20 2020-05-24T13:47:04.907 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 11/20 2020-05-24T13:47:05.004 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-mon-14.2.9-537.g6fe217b2dd.el7.x86_64 8/20 2020-05-24T13:47:05.120 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 9/20 2020-05-24T13:47:05.228 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 10/20 2020-05-24T13:47:05.301 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-mon-14.2.9-537.g6fe217b2dd.el7.x86_64 12/20 2020-05-24T13:47:05.365 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 11/20 2020-05-24T13:47:05.463 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 12/20 2020-05-24T13:47:05.555 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-mds-14.2.9-537.g6fe217b2dd.el7.x86_64 13/20 2020-05-24T13:47:05.590 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-six-1.9.0-0.el7.noarch 13/20 2020-05-24T13:47:05.848 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-osd-14.2.9-537.g6fe217b2dd.el7.x86_64 14/20 2020-05-24T13:47:05.848 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 15/20 2020-05-24T13:47:05.848 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 16/20 2020-05-24T13:47:05.915 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 17/20 2020-05-24T13:47:06.025 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 18/20 2020-05-24T13:47:06.064 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 14/20 2020-05-24T13:47:06.167 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 19/20 2020-05-24T13:47:06.380 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 15/20 2020-05-24T13:47:06.638 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 16/20 2020-05-24T13:47:06.694 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 20/20 2020-05-24T13:47:06.695 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:06.695 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:06.695 INFO:teuthology.orchestra.run.smithi160.stdout: ceph.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:06.695 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:06.696 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Installed: 2020-05-24T13:47:06.696 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:06.696 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:06.696 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:06.696 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:06.697 INFO:teuthology.orchestra.run.smithi160.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2020-05-24T13:47:06.697 INFO:teuthology.orchestra.run.smithi160.stdout: python-beaker.noarch 0:1.5.4-10.el7 2020-05-24T13:47:06.697 INFO:teuthology.orchestra.run.smithi160.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2020-05-24T13:47:06.697 INFO:teuthology.orchestra.run.smithi160.stdout: python-mako.noarch 0:0.8.1-2.el7 2020-05-24T13:47:06.697 INFO:teuthology.orchestra.run.smithi160.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2020-05-24T13:47:06.698 INFO:teuthology.orchestra.run.smithi160.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2020-05-24T13:47:06.698 INFO:teuthology.orchestra.run.smithi160.stdout: python-pecan.noarch 0:0.4.5-2.el7 2020-05-24T13:47:06.698 INFO:teuthology.orchestra.run.smithi160.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2020-05-24T13:47:06.698 INFO:teuthology.orchestra.run.smithi160.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2020-05-24T13:47:06.698 INFO:teuthology.orchestra.run.smithi160.stdout: python-tempita.noarch 0:0.5.1-6.el7 2020-05-24T13:47:06.699 INFO:teuthology.orchestra.run.smithi160.stdout: python-webob.noarch 0:1.2.3-7.el7 2020-05-24T13:47:06.699 INFO:teuthology.orchestra.run.smithi160.stdout: python-webtest.noarch 0:1.3.4-6.el7 2020-05-24T13:47:06.699 INFO:teuthology.orchestra.run.smithi160.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2020-05-24T13:47:06.699 INFO:teuthology.orchestra.run.smithi160.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2020-05-24T13:47:06.699 INFO:teuthology.orchestra.run.smithi160.stdout: python2-six.noarch 0:1.9.0-0.el7 2020-05-24T13:47:06.699 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:06.700 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:06.814 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr 2020-05-24T13:47:07.054 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 17/20 2020-05-24T13:47:07.057 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:07.098 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:07.098 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:07.099 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:07.099 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:07.100 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:07.734 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 18/20 2020-05-24T13:47:07.740 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:08.066 INFO:teuthology.orchestra.run.smithi160.stdout:Package 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:08.066 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do 2020-05-24T13:47:08.135 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-dashboard 2020-05-24T13:47:08.169 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 19/20 2020-05-24T13:47:08.397 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:08.442 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:08.443 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:08.444 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:08.444 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:08.444 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:08.465 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 20/20 2020-05-24T13:47:08.584 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 1/20 2020-05-24T13:47:08.718 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 2/20 2020-05-24T13:47:08.860 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 3/20 2020-05-24T13:47:09.089 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:09.128 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 4/20 2020-05-24T13:47:09.238 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 5/20 2020-05-24T13:47:09.328 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 6/20 2020-05-24T13:47:09.421 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 7/20 2020-05-24T13:47:09.425 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:09.426 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:09.426 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-mgr-dashboard.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:09.429 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:09.460 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:09.461 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:09.461 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-grafana-dashboards.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:09.462 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2020-05-24T13:47:09.462 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2020-05-24T13:47:09.463 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:09.463 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2020-05-24T13:47:09.530 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-mon-14.2.9-537.g6fe217b2dd.el7.x86_64 8/20 2020-05-24T13:47:09.630 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 9/20 2020-05-24T13:47:09.764 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 10/20 2020-05-24T13:47:09.831 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:09.881 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 11/20 2020-05-24T13:47:10.040 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 12/20 2020-05-24T13:47:10.052 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:10.053 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:47:10.054 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:10.054 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:10.054 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2020-05-24T13:47:10.055 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:10.055 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:47:10.055 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 4.0 M 2020-05-24T13:47:10.055 INFO:teuthology.orchestra.run.smithi160.stdout:Installing for dependencies: 2020-05-24T13:47:10.056 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 18 k 2020-05-24T13:47:10.056 INFO:teuthology.orchestra.run.smithi160.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2020-05-24T13:47:10.056 INFO:teuthology.orchestra.run.smithi160.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2020-05-24T13:47:10.056 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:10.056 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:47:10.057 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:10.057 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package (+3 Dependent packages) 2020-05-24T13:47:10.057 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:10.057 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 4.6 M 2020-05-24T13:47:10.058 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 20 M 2020-05-24T13:47:10.058 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:47:10.186 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-mds-14.2.9-537.g6fe217b2dd.el7.x86_64 13/20 2020-05-24T13:47:10.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-osd-14.2.9-537.g6fe217b2dd.el7.x86_64 14/20 2020-05-24T13:47:10.426 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 15/20 2020-05-24T13:47:10.535 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 16/20 2020-05-24T13:47:10.677 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 17/20 2020-05-24T13:47:10.769 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 18/20 2020-05-24T13:47:10.774 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:10.775 INFO:teuthology.orchestra.run.smithi160.stdout:Total 6.4 MB/s | 4.6 MB 00:00 2020-05-24T13:47:10.779 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:47:10.788 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:47:10.853 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 19/20 2020-05-24T13:47:10.954 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:47:10.955 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:47:11.298 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-14.2.9-537.g6fe217b2dd.el7.x86_64 20/20 2020-05-24T13:47:11.298 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:11.299 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:47:11.299 INFO:teuthology.orchestra.run.smithi050.stdout: ceph.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:11.299 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:11.299 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Installed: 2020-05-24T13:47:11.299 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:11.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:11.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:11.300 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:11.300 INFO:teuthology.orchestra.run.smithi050.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2020-05-24T13:47:11.301 INFO:teuthology.orchestra.run.smithi050.stdout: python-beaker.noarch 0:1.5.4-10.el7 2020-05-24T13:47:11.301 INFO:teuthology.orchestra.run.smithi050.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2020-05-24T13:47:11.301 INFO:teuthology.orchestra.run.smithi050.stdout: python-mako.noarch 0:0.8.1-2.el7 2020-05-24T13:47:11.301 INFO:teuthology.orchestra.run.smithi050.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2020-05-24T13:47:11.301 INFO:teuthology.orchestra.run.smithi050.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2020-05-24T13:47:11.302 INFO:teuthology.orchestra.run.smithi050.stdout: python-pecan.noarch 0:0.4.5-2.el7 2020-05-24T13:47:11.302 INFO:teuthology.orchestra.run.smithi050.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2020-05-24T13:47:11.302 INFO:teuthology.orchestra.run.smithi050.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2020-05-24T13:47:11.302 INFO:teuthology.orchestra.run.smithi050.stdout: python-tempita.noarch 0:0.5.1-6.el7 2020-05-24T13:47:11.303 INFO:teuthology.orchestra.run.smithi050.stdout: python-webob.noarch 0:1.2.3-7.el7 2020-05-24T13:47:11.303 INFO:teuthology.orchestra.run.smithi050.stdout: python-webtest.noarch 0:1.3.4-6.el7 2020-05-24T13:47:11.303 INFO:teuthology.orchestra.run.smithi050.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2020-05-24T13:47:11.303 INFO:teuthology.orchestra.run.smithi050.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2020-05-24T13:47:11.303 INFO:teuthology.orchestra.run.smithi050.stdout: python2-six.noarch 0:1.9.0-0.el7 2020-05-24T13:47:11.304 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:11.304 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:47:11.415 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr 2020-05-24T13:47:11.627 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:11.637 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-grafana-dashboards-14.2.9-537.g6fe217b2dd.el7.noarc 1/4 2020-05-24T13:47:11.672 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:11.672 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:11.672 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:11.672 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:11.673 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:12.056 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2020-05-24T13:47:12.312 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:12.649 INFO:teuthology.orchestra.run.smithi050.stdout:Package 2:ceph-mgr-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:12.650 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do 2020-05-24T13:47:12.711 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-dashboard 2020-05-24T13:47:12.877 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:12.914 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:12.914 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:12.915 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:12.915 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:12.916 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:13.192 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2020-05-24T13:47:13.530 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:13.556 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:13.697 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2020-05-24T13:47:13.806 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-grafana-dashboards-14.2.9-537.g6fe217b2dd.el7.noarc 2/4 2020-05-24T13:47:13.865 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:47:13.866 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:13.866 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-mgr-dashboard.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:13.870 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:13.906 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:13.907 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:13.907 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-grafana-dashboards.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:13.908 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2020-05-24T13:47:13.908 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2020-05-24T13:47:13.908 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:13.909 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2020-05-24T13:47:13.993 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-routes-1.13-2.el7.noarch 3/4 2020-05-24T13:47:14.297 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:14.525 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:14.525 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:47:14.526 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:14.526 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:14.526 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2020-05-24T13:47:14.526 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:14.527 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:47:14.527 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 4.0 M 2020-05-24T13:47:14.527 INFO:teuthology.orchestra.run.smithi050.stdout:Installing for dependencies: 2020-05-24T13:47:14.527 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 18 k 2020-05-24T13:47:14.527 INFO:teuthology.orchestra.run.smithi050.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2020-05-24T13:47:14.528 INFO:teuthology.orchestra.run.smithi050.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2020-05-24T13:47:14.528 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:14.528 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:47:14.528 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:14.528 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package (+3 Dependent packages) 2020-05-24T13:47:14.529 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:14.529 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 4.6 M 2020-05-24T13:47:14.529 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 20 M 2020-05-24T13:47:14.529 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:47:14.550 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:14.550 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:14.551 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:14.551 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:14.551 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:14.551 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Installed: 2020-05-24T13:47:14.551 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:14.552 INFO:teuthology.orchestra.run.smithi160.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2020-05-24T13:47:14.552 INFO:teuthology.orchestra.run.smithi160.stdout: python-routes.noarch 0:1.13-2.el7 2020-05-24T13:47:14.552 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:14.552 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:14.679 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-diskprediction-cloud 2020-05-24T13:47:14.891 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:14.931 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:14.932 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:14.933 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:14.933 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:14.934 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:15.602 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:15.901 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:15.902 INFO:teuthology.orchestra.run.smithi050.stdout:Total 3.4 MB/s | 4.6 MB 00:01 2020-05-24T13:47:15.908 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:47:15.918 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:47:15.928 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:15.929 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:15.929 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:16.106 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:47:16.107 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:47:16.298 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:16.482 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:16.483 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:47:16.483 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:16.484 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:16.484 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2020-05-24T13:47:16.484 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:16.484 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:47:16.484 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-cloud 2020-05-24T13:47:16.485 INFO:teuthology.orchestra.run.smithi160.stdout: noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 83 k 2020-05-24T13:47:16.485 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:16.485 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:47:16.485 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:16.485 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2020-05-24T13:47:16.486 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:16.486 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 83 k 2020-05-24T13:47:16.486 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 412 k 2020-05-24T13:47:16.487 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:47:16.825 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-grafana-dashboards-14.2.9-537.g6fe217b2dd.el7.noarc 1/4 2020-05-24T13:47:16.915 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:47:16.918 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:47:16.958 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:47:16.959 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:47:17.265 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2020-05-24T13:47:17.764 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-14.2.9-537.g6fe217b2dd.el7 1/1 2020-05-24T13:47:18.179 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-14.2.9-537.g6fe217b2dd.el7 1/1 2020-05-24T13:47:18.180 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:18.180 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:18.180 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-cloud.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:18.180 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:18.180 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:18.308 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-diskprediction-local 2020-05-24T13:47:18.390 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2020-05-24T13:47:18.541 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:18.584 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:18.585 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:18.586 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:18.586 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:18.586 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:18.800 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:18.908 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2020-05-24T13:47:19.231 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:19.301 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-grafana-dashboards-14.2.9-537.g6fe217b2dd.el7.noarc 2/4 2020-05-24T13:47:19.409 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-routes-1.13-2.el7.noarch 3/4 2020-05-24T13:47:19.568 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:19.569 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:19.569 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:19.571 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:19.603 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:19.604 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2020-05-24T13:47:19.610 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:19.611 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:19.613 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:19.613 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:19.614 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2020-05-24T13:47:19.617 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2020-05-24T13:47:19.617 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2020-05-24T13:47:19.617 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2020-05-24T13:47:19.618 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:19.618 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2020-05-24T13:47:19.847 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-mgr-dashboard-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:19.848 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:19.848 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:47:19.848 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:19.848 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:19.849 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Installed: 2020-05-24T13:47:19.849 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:19.849 INFO:teuthology.orchestra.run.smithi050.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2020-05-24T13:47:19.849 INFO:teuthology.orchestra.run.smithi050.stdout: python-routes.noarch 0:1.13-2.el7 2020-05-24T13:47:19.849 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:19.850 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:47:19.981 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:19.984 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-diskprediction-cloud 2020-05-24T13:47:20.206 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:20.206 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:47:20.207 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:20.207 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:20.208 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2020-05-24T13:47:20.208 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:20.208 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:47:20.208 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local 2020-05-24T13:47:20.209 INFO:teuthology.orchestra.run.smithi160.stdout: noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 1.1 M 2020-05-24T13:47:20.209 INFO:teuthology.orchestra.run.smithi160.stdout:Installing for dependencies: 2020-05-24T13:47:20.209 INFO:teuthology.orchestra.run.smithi160.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2020-05-24T13:47:20.209 INFO:teuthology.orchestra.run.smithi160.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2020-05-24T13:47:20.209 INFO:teuthology.orchestra.run.smithi160.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2020-05-24T13:47:20.210 INFO:teuthology.orchestra.run.smithi160.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2020-05-24T13:47:20.210 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:20.210 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:47:20.210 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:20.210 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package (+4 Dependent packages) 2020-05-24T13:47:20.211 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:20.211 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 12 M 2020-05-24T13:47:20.211 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 66 M 2020-05-24T13:47:20.211 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:47:20.264 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:20.311 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:20.312 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:20.313 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:20.313 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:20.314 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:20.724 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:20.724 INFO:teuthology.orchestra.run.smithi160.stdout:Total 23 MB/s | 12 MB 00:00 2020-05-24T13:47:20.734 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:47:20.756 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:47:20.864 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:47:20.864 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:47:20.968 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:21.321 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:47:21.322 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:21.322 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:21.711 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:21.720 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2020-05-24T13:47:21.900 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:21.901 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:47:21.901 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:21.902 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:21.902 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2020-05-24T13:47:21.902 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:21.902 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:47:21.903 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-cloud 2020-05-24T13:47:21.903 INFO:teuthology.orchestra.run.smithi050.stdout: noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 83 k 2020-05-24T13:47:21.903 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:21.903 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:47:21.903 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:21.903 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2020-05-24T13:47:21.904 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:21.904 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 83 k 2020-05-24T13:47:21.904 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 412 k 2020-05-24T13:47:21.906 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:47:22.269 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2020-05-24T13:47:22.333 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:47:22.336 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:47:22.387 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:47:22.388 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:47:23.125 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-14.2.9-537.g6fe217b2dd.el7 1/1 2020-05-24T13:47:23.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-14.2.9-537.g6fe217b2dd.el7 1/1 2020-05-24T13:47:23.615 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:23.615 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:47:23.616 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-cloud.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:23.616 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:23.616 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:47:23.740 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-diskprediction-local 2020-05-24T13:47:23.947 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:23.987 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2020-05-24T13:47:23.987 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:23.989 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:23.990 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:23.990 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:23.990 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:24.654 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:24.990 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:47:24.991 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:24.991 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:24.994 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:25.028 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:25.028 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2020-05-24T13:47:25.034 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:25.035 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:25.038 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2020-05-24T13:47:25.038 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:25.039 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2020-05-24T13:47:25.039 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2020-05-24T13:47:25.040 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2020-05-24T13:47:25.041 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2020-05-24T13:47:25.042 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:25.042 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2020-05-24T13:47:25.184 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2020-05-24T13:47:25.426 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:25.500 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7 5/5 2020-05-24T13:47:25.633 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2020-05-24T13:47:25.657 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:25.657 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:47:25.658 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:25.658 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:25.659 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2020-05-24T13:47:25.659 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:25.659 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:47:25.659 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local 2020-05-24T13:47:25.659 INFO:teuthology.orchestra.run.smithi050.stdout: noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 1.1 M 2020-05-24T13:47:25.660 INFO:teuthology.orchestra.run.smithi050.stdout:Installing for dependencies: 2020-05-24T13:47:25.660 INFO:teuthology.orchestra.run.smithi050.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2020-05-24T13:47:25.660 INFO:teuthology.orchestra.run.smithi050.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2020-05-24T13:47:25.660 INFO:teuthology.orchestra.run.smithi050.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2020-05-24T13:47:25.660 INFO:teuthology.orchestra.run.smithi050.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2020-05-24T13:47:25.661 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:25.661 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:47:25.661 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:25.661 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package (+4 Dependent packages) 2020-05-24T13:47:25.661 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:25.661 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 12 M 2020-05-24T13:47:25.662 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 66 M 2020-05-24T13:47:25.662 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:47:25.742 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 2/5 2020-05-24T13:47:25.851 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2020-05-24T13:47:25.960 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7 4/5 2020-05-24T13:47:26.206 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:26.207 INFO:teuthology.orchestra.run.smithi050.stdout:Total 21 MB/s | 12 MB 00:00 2020-05-24T13:47:26.224 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:47:26.262 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:47:26.404 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2020-05-24T13:47:26.404 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:26.404 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:26.405 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:26.405 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:26.406 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Installed: 2020-05-24T13:47:26.406 INFO:teuthology.orchestra.run.smithi160.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2020-05-24T13:47:26.406 INFO:teuthology.orchestra.run.smithi160.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2020-05-24T13:47:26.407 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:26.407 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:26.512 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:47:26.513 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:47:26.529 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-rook 2020-05-24T13:47:26.787 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:26.831 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:26.831 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:26.832 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:26.832 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:26.833 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:27.408 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2020-05-24T13:47:27.475 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:27.810 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:27.811 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:27.811 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-mgr-rook.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:27.812 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:27.844 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:27.845 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-kubernetes.noarch 0:11.0.0-2.el7 will be installed 2020-05-24T13:47:27.847 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:27.848 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:27.849 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:27.850 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:27.851 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:27.851 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2020-05-24T13:47:27.851 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2020-05-24T13:47:27.852 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2020-05-24T13:47:27.852 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:27.853 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:27.854 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:27.855 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2020-05-24T13:47:27.855 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2020-05-24T13:47:27.856 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:27.856 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2020-05-24T13:47:27.857 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2020-05-24T13:47:27.857 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2020-05-24T13:47:27.858 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2020-05-24T13:47:27.858 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-rsa.noarch 0:3.4.1-3.el7 will be installed 2020-05-24T13:47:27.858 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:27.859 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2020-05-24T13:47:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2020-05-24T13:47:28.233 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:28.453 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:28.454 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:47:28.455 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:28.455 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:28.456 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2020-05-24T13:47:28.456 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:28.456 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:47:28.456 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 21 k 2020-05-24T13:47:28.456 INFO:teuthology.orchestra.run.smithi160.stdout:Installing for dependencies: 2020-05-24T13:47:28.457 INFO:teuthology.orchestra.run.smithi160.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2020-05-24T13:47:28.457 INFO:teuthology.orchestra.run.smithi160.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2020-05-24T13:47:28.457 INFO:teuthology.orchestra.run.smithi160.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2020-05-24T13:47:28.457 INFO:teuthology.orchestra.run.smithi160.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2020-05-24T13:47:28.457 INFO:teuthology.orchestra.run.smithi160.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2020-05-24T13:47:28.458 INFO:teuthology.orchestra.run.smithi160.stdout: python2-kubernetes noarch 11.0.0-2.el7 epel 1.1 M 2020-05-24T13:47:28.458 INFO:teuthology.orchestra.run.smithi160.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2020-05-24T13:47:28.458 INFO:teuthology.orchestra.run.smithi160.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2020-05-24T13:47:28.458 INFO:teuthology.orchestra.run.smithi160.stdout: python2-requests-oauthlib 2020-05-24T13:47:28.458 INFO:teuthology.orchestra.run.smithi160.stdout: noarch 0.8.0-5.el7 base 35 k 2020-05-24T13:47:28.459 INFO:teuthology.orchestra.run.smithi160.stdout: python2-rsa noarch 3.4.1-3.el7 epel 68 k 2020-05-24T13:47:28.459 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:28.459 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:47:28.459 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:28.459 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package (+10 Dependent packages) 2020-05-24T13:47:28.460 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:28.460 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 1.7 M 2020-05-24T13:47:28.460 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 25 M 2020-05-24T13:47:28.461 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:47:28.821 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:28.822 INFO:teuthology.orchestra.run.smithi160.stdout:Total 4.6 MB/s | 1.7 MB 00:00 2020-05-24T13:47:28.837 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:47:28.850 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:47:28.926 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:47:28.926 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:47:29.840 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2020-05-24T13:47:30.171 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-rsa-3.4.1-3.el7.noarch 1/11 2020-05-24T13:47:30.596 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2020-05-24T13:47:30.662 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2020-05-24T13:47:30.946 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2020-05-24T13:47:30.990 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7 5/5 2020-05-24T13:47:31.103 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2020-05-24T13:47:31.220 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 2/5 2020-05-24T13:47:31.283 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2020-05-24T13:47:31.359 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2020-05-24T13:47:31.479 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-mgr-diskprediction-local-14.2.9-537.g6fe217b2dd.el7 4/5 2020-05-24T13:47:31.536 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 5/11 2020-05-24T13:47:31.876 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 6/11 2020-05-24T13:47:32.007 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2020-05-24T13:47:32.007 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:32.007 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:47:32.008 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:32.008 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:32.008 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Installed: 2020-05-24T13:47:32.008 INFO:teuthology.orchestra.run.smithi050.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2020-05-24T13:47:32.008 INFO:teuthology.orchestra.run.smithi050.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2020-05-24T13:47:32.009 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:32.009 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:47:32.137 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-rook 2020-05-24T13:47:32.232 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 7/11 2020-05-24T13:47:32.383 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:32.426 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:32.427 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:32.427 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:32.428 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:32.428 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:32.588 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2020-05-24T13:47:33.093 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:33.437 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:47:33.437 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:33.438 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-mgr-rook.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:33.439 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:33.472 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:33.472 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-kubernetes.noarch 0:11.0.0-2.el7 will be installed 2020-05-24T13:47:33.475 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:33.476 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:33.477 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:33.478 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-11.0.0-2.el7.noarch 2020-05-24T13:47:33.479 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:33.479 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2020-05-24T13:47:33.480 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2020-05-24T13:47:33.480 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2020-05-24T13:47:33.481 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:33.482 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:33.483 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2020-05-24T13:47:33.484 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2020-05-24T13:47:33.484 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2020-05-24T13:47:33.485 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:33.485 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2020-05-24T13:47:33.486 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2020-05-24T13:47:33.486 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2020-05-24T13:47:33.487 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2020-05-24T13:47:33.487 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-rsa.noarch 0:3.4.1-3.el7 will be installed 2020-05-24T13:47:33.487 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:33.488 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2020-05-24T13:47:33.874 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:33.891 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2020-05-24T13:47:34.108 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:34.108 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:47:34.110 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:34.110 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:34.110 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2020-05-24T13:47:34.110 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:34.111 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:47:34.111 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 21 k 2020-05-24T13:47:34.111 INFO:teuthology.orchestra.run.smithi050.stdout:Installing for dependencies: 2020-05-24T13:47:34.111 INFO:teuthology.orchestra.run.smithi050.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2020-05-24T13:47:34.111 INFO:teuthology.orchestra.run.smithi050.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2020-05-24T13:47:34.111 INFO:teuthology.orchestra.run.smithi050.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2020-05-24T13:47:34.112 INFO:teuthology.orchestra.run.smithi050.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2020-05-24T13:47:34.112 INFO:teuthology.orchestra.run.smithi050.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2020-05-24T13:47:34.112 INFO:teuthology.orchestra.run.smithi050.stdout: python2-kubernetes noarch 11.0.0-2.el7 epel 1.1 M 2020-05-24T13:47:34.112 INFO:teuthology.orchestra.run.smithi050.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2020-05-24T13:47:34.112 INFO:teuthology.orchestra.run.smithi050.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2020-05-24T13:47:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: python2-requests-oauthlib 2020-05-24T13:47:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: noarch 0.8.0-5.el7 base 35 k 2020-05-24T13:47:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: python2-rsa noarch 3.4.1-3.el7 epel 68 k 2020-05-24T13:47:34.113 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:34.113 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:47:34.114 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:34.114 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package (+10 Dependent packages) 2020-05-24T13:47:34.114 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:34.115 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 1.7 M 2020-05-24T13:47:34.115 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 25 M 2020-05-24T13:47:34.115 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:47:34.225 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-kubernetes-11.0.0-2.el7.noarch 10/11 2020-05-24T13:47:34.478 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:34.478 INFO:teuthology.orchestra.run.smithi050.stdout:Total 4.6 MB/s | 1.7 MB 00:00 2020-05-24T13:47:34.490 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:47:34.501 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:47:34.564 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:47:34.565 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:47:34.864 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 11/11 2020-05-24T13:47:34.979 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2020-05-24T13:47:35.113 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2020-05-24T13:47:35.222 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 3/11 2020-05-24T13:47:35.282 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-rsa-3.4.1-3.el7.noarch 1/11 2020-05-24T13:47:35.339 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2020-05-24T13:47:35.481 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2020-05-24T13:47:35.590 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2020-05-24T13:47:35.686 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2020-05-24T13:47:35.699 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2020-05-24T13:47:35.832 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 8/11 2020-05-24T13:47:35.916 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2020-05-24T13:47:36.009 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-rsa-3.4.1-3.el7.noarch 10/11 2020-05-24T13:47:36.043 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2020-05-24T13:47:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2020-05-24T13:47:36.436 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-kubernetes-11.0.0-2.el7.noarch 11/11 2020-05-24T13:47:36.436 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:36.436 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:36.436 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:36.437 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:36.437 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Installed: 2020-05-24T13:47:36.437 INFO:teuthology.orchestra.run.smithi160.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2020-05-24T13:47:36.437 INFO:teuthology.orchestra.run.smithi160.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2020-05-24T13:47:36.437 INFO:teuthology.orchestra.run.smithi160.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2020-05-24T13:47:36.438 INFO:teuthology.orchestra.run.smithi160.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2020-05-24T13:47:36.438 INFO:teuthology.orchestra.run.smithi160.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2020-05-24T13:47:36.438 INFO:teuthology.orchestra.run.smithi160.stdout: python2-kubernetes.noarch 0:11.0.0-2.el7 2020-05-24T13:47:36.438 INFO:teuthology.orchestra.run.smithi160.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2020-05-24T13:47:36.438 INFO:teuthology.orchestra.run.smithi160.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2020-05-24T13:47:36.439 INFO:teuthology.orchestra.run.smithi160.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2020-05-24T13:47:36.439 INFO:teuthology.orchestra.run.smithi160.stdout: python2-rsa.noarch 0:3.4.1-3.el7 2020-05-24T13:47:36.439 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:36.439 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:36.587 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-mgr-ssh 2020-05-24T13:47:36.972 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 5/11 2020-05-24T13:47:36.996 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 6/11 2020-05-24T13:47:37.228 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:37.270 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:37.271 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:37.272 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:37.272 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:37.273 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:37.327 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 7/11 2020-05-24T13:47:37.622 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2020-05-24T13:47:37.906 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:38.242 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:38.242 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:38.243 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-mgr-ssh.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:38.244 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:38.276 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:38.276 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-remoto.noarch 0:1.0.0-1.el7 will be installed 2020-05-24T13:47:38.277 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python2-execnet >= 1.2.0 for package: python2-remoto-1.0.0-1.el7.noarch 2020-05-24T13:47:38.280 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:38.280 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-execnet.noarch 0:1.7.1-3.el7 will be installed 2020-05-24T13:47:38.280 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: python-apipkg for package: python-execnet-1.7.1-3.el7.noarch 2020-05-24T13:47:38.280 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:38.280 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python2-apipkg.noarch 0:1.5-4.el7 will be installed 2020-05-24T13:47:38.651 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:38.878 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:38.878 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:47:38.879 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:38.879 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:38.879 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2020-05-24T13:47:38.880 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:38.880 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:47:38.880 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-ssh noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 21 k 2020-05-24T13:47:38.880 INFO:teuthology.orchestra.run.smithi160.stdout:Installing for dependencies: 2020-05-24T13:47:38.880 INFO:teuthology.orchestra.run.smithi160.stdout: python-execnet noarch 1.7.1-3.el7 epel 260 k 2020-05-24T13:47:38.881 INFO:teuthology.orchestra.run.smithi160.stdout: python2-apipkg noarch 1.5-4.el7 epel 17 k 2020-05-24T13:47:38.881 INFO:teuthology.orchestra.run.smithi160.stdout: python2-remoto noarch 1.0.0-1.el7 epel 42 k 2020-05-24T13:47:38.881 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:38.881 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:47:38.881 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:38.882 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package (+3 Dependent packages) 2020-05-24T13:47:38.882 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:38.882 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 340 k 2020-05-24T13:47:38.882 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 1.1 M 2020-05-24T13:47:38.882 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:47:38.960 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2020-05-24T13:47:39.536 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:39.537 INFO:teuthology.orchestra.run.smithi160.stdout:Total 517 kB/s | 340 kB 00:00 2020-05-24T13:47:39.541 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:47:39.546 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:47:39.591 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:47:39.592 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:47:39.612 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-kubernetes-11.0.0-2.el7.noarch 10/11 2020-05-24T13:47:39.927 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 11/11 2020-05-24T13:47:40.042 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2020-05-24T13:47:40.159 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2020-05-24T13:47:40.242 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-mgr-rook-14.2.9-537.g6fe217b2dd.el7.noarch 3/11 2020-05-24T13:47:40.343 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2020-05-24T13:47:40.384 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-apipkg-1.5-4.el7.noarch 1/4 2020-05-24T13:47:40.485 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2020-05-24T13:47:40.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2020-05-24T13:47:40.719 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2020-05-24T13:47:40.839 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python-execnet-1.7.1-3.el7.noarch 2/4 2020-05-24T13:47:40.861 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 8/11 2020-05-24T13:47:40.970 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2020-05-24T13:47:41.087 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-rsa-3.4.1-3.el7.noarch 10/11 2020-05-24T13:47:41.233 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python2-remoto-1.0.0-1.el7.noarch 3/4 2020-05-24T13:47:41.510 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:41.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-kubernetes-11.0.0-2.el7.noarch 11/11 2020-05-24T13:47:41.603 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:41.603 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:47:41.604 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:41.604 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:41.604 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Installed: 2020-05-24T13:47:41.605 INFO:teuthology.orchestra.run.smithi050.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2020-05-24T13:47:41.605 INFO:teuthology.orchestra.run.smithi050.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2020-05-24T13:47:41.605 INFO:teuthology.orchestra.run.smithi050.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2020-05-24T13:47:41.606 INFO:teuthology.orchestra.run.smithi050.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2020-05-24T13:47:41.606 INFO:teuthology.orchestra.run.smithi050.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2020-05-24T13:47:41.606 INFO:teuthology.orchestra.run.smithi050.stdout: python2-kubernetes.noarch 0:11.0.0-2.el7 2020-05-24T13:47:41.607 INFO:teuthology.orchestra.run.smithi050.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2020-05-24T13:47:41.607 INFO:teuthology.orchestra.run.smithi050.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2020-05-24T13:47:41.607 INFO:teuthology.orchestra.run.smithi050.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2020-05-24T13:47:41.608 INFO:teuthology.orchestra.run.smithi050.stdout: python2-rsa.noarch 0:3.4.1-3.el7 2020-05-24T13:47:41.608 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:41.608 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:47:41.625 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python-execnet-1.7.1-3.el7.noarch 1/4 2020-05-24T13:47:41.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-remoto-1.0.0-1.el7.noarch 2/4 2020-05-24T13:47:41.729 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-ssh 2020-05-24T13:47:41.793 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python2-apipkg-1.5-4.el7.noarch 3/4 2020-05-24T13:47:41.972 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:42.011 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:42.012 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:42.013 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:42.013 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:42.013 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:42.336 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:42.337 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:42.337 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:42.337 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-ssh.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:42.338 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:42.338 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Installed: 2020-05-24T13:47:42.338 INFO:teuthology.orchestra.run.smithi160.stdout: python-execnet.noarch 0:1.7.1-3.el7 python2-apipkg.noarch 0:1.5-4.el7 2020-05-24T13:47:42.338 INFO:teuthology.orchestra.run.smithi160.stdout: python2-remoto.noarch 0:1.0.0-1.el7 2020-05-24T13:47:42.338 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:42.339 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:42.468 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-fuse 2020-05-24T13:47:42.672 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:42.726 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:42.770 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:42.771 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:42.771 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:42.771 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:42.772 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:43.009 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:47:43.010 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:43.010 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-mgr-ssh.noarch 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:43.011 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 2020-05-24T13:47:43.045 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:43.045 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-remoto.noarch 0:1.0.0-1.el7 will be installed 2020-05-24T13:47:43.046 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python2-execnet >= 1.2.0 for package: python2-remoto-1.0.0-1.el7.noarch 2020-05-24T13:47:43.047 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:43.047 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-execnet.noarch 0:1.7.1-3.el7 will be installed 2020-05-24T13:47:43.047 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: python-apipkg for package: python-execnet-1.7.1-3.el7.noarch 2020-05-24T13:47:43.048 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:43.048 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python2-apipkg.noarch 0:1.5-4.el7 will be installed 2020-05-24T13:47:43.427 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:43.439 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:43.671 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:43.671 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:47:43.672 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:43.672 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:43.672 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2020-05-24T13:47:43.673 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:43.673 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:47:43.673 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-ssh noarch 2:14.2.9-537.g6fe217b2dd.el7 ceph-noarch 21 k 2020-05-24T13:47:43.673 INFO:teuthology.orchestra.run.smithi050.stdout:Installing for dependencies: 2020-05-24T13:47:43.673 INFO:teuthology.orchestra.run.smithi050.stdout: python-execnet noarch 1.7.1-3.el7 epel 260 k 2020-05-24T13:47:43.674 INFO:teuthology.orchestra.run.smithi050.stdout: python2-apipkg noarch 1.5-4.el7 epel 17 k 2020-05-24T13:47:43.674 INFO:teuthology.orchestra.run.smithi050.stdout: python2-remoto noarch 1.0.0-1.el7 epel 42 k 2020-05-24T13:47:43.674 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:43.674 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:47:43.674 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:43.675 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package (+3 Dependent packages) 2020-05-24T13:47:43.675 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:43.675 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 340 k 2020-05-24T13:47:43.675 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 1.1 M 2020-05-24T13:47:43.675 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:47:43.758 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:43.759 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:43.760 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package ceph-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:44.140 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:44.289 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:44.290 INFO:teuthology.orchestra.run.smithi050.stdout:Total 552 kB/s | 340 kB 00:00 2020-05-24T13:47:44.294 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:47:44.300 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:47:44.328 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:44.328 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:47:44.329 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:44.329 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:44.329 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository 2020-05-24T13:47:44.329 INFO:teuthology.orchestra.run.smithi160.stdout: Size 2020-05-24T13:47:44.330 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:44.330 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:47:44.330 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 582 k 2020-05-24T13:47:44.330 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:44.330 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:47:44.331 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:44.331 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2020-05-24T13:47:44.331 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:44.331 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 582 k 2020-05-24T13:47:44.331 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 1.6 M 2020-05-24T13:47:44.332 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:47:44.342 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:47:44.342 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:47:45.084 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-apipkg-1.5-4.el7.noarch 1/4 2020-05-24T13:47:45.492 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python-execnet-1.7.1-3.el7.noarch 2/4 2020-05-24T13:47:45.499 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:47:45.526 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:47:45.528 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:47:45.529 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:47:45.854 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python2-remoto-1.0.0-1.el7.noarch 3/4 2020-05-24T13:47:46.176 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:46.296 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python-execnet-1.7.1-3.el7.noarch 1/4 2020-05-24T13:47:46.298 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:ceph-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:47:46.396 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-remoto-1.0.0-1.el7.noarch 2/4 2020-05-24T13:47:46.532 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python2-apipkg-1.5-4.el7.noarch 3/4 2020-05-24T13:47:46.715 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:ceph-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:47:46.716 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:46.716 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:46.716 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:46.716 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:46.717 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:46.832 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install libcephfs2 2020-05-24T13:47:47.041 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-mgr-ssh-14.2.9-537.g6fe217b2dd.el7.noarch 4/4 2020-05-24T13:47:47.041 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:47.041 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:47:47.042 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-ssh.noarch 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:47.042 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:47.042 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Installed: 2020-05-24T13:47:47.042 INFO:teuthology.orchestra.run.smithi050.stdout: python-execnet.noarch 0:1.7.1-3.el7 python2-apipkg.noarch 0:1.5-4.el7 2020-05-24T13:47:47.042 INFO:teuthology.orchestra.run.smithi050.stdout: python2-remoto.noarch 0:1.0.0-1.el7 2020-05-24T13:47:47.043 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:47.043 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:47:47.062 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:47.103 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:47.104 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:47.104 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:47.104 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:47.105 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:47.170 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-fuse 2020-05-24T13:47:47.438 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:47.486 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:47.487 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:47.487 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:47.488 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:47.488 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:47.747 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:48.078 INFO:teuthology.orchestra.run.smithi160.stdout:Package 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:48.079 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do 2020-05-24T13:47:48.140 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install libcephfs-devel 2020-05-24T13:47:48.156 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:48.376 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:48.421 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:48.422 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:48.423 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:48.424 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:48.424 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:48.513 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:47:48.514 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:48.514 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package ceph-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:48.927 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:49.077 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:49.120 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:49.120 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:47:49.121 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:49.122 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:49.122 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository 2020-05-24T13:47:49.122 INFO:teuthology.orchestra.run.smithi050.stdout: Size 2020-05-24T13:47:49.122 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:49.122 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:47:49.123 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 582 k 2020-05-24T13:47:49.123 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:49.123 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:47:49.123 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:49.123 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2020-05-24T13:47:49.123 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:49.124 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 582 k 2020-05-24T13:47:49.124 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 1.6 M 2020-05-24T13:47:49.124 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:47:49.415 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:49.416 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:49.416 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package libcephfs-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:49.417 INFO:teuthology.orchestra.run.smithi160.stdout:--> Processing Dependency: librados-devel = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:49.449 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:49.450 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package librados-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:49.831 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:50.056 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:50.057 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:47:50.058 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:50.058 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:50.058 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository 2020-05-24T13:47:50.058 INFO:teuthology.orchestra.run.smithi160.stdout: Size 2020-05-24T13:47:50.059 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:50.059 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:47:50.059 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 17 k 2020-05-24T13:47:50.060 INFO:teuthology.orchestra.run.smithi160.stdout:Installing for dependencies: 2020-05-24T13:47:50.060 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 92 k 2020-05-24T13:47:50.060 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:50.060 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:47:50.061 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:50.061 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package (+1 Dependent package) 2020-05-24T13:47:50.061 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:50.061 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 109 k 2020-05-24T13:47:50.062 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 391 k 2020-05-24T13:47:50.062 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:47:50.359 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:47:50.387 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:47:50.389 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:47:50.390 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:47:50.650 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:50.651 INFO:teuthology.orchestra.run.smithi160.stdout:Total 184 kB/s | 109 kB 00:00 2020-05-24T13:47:50.653 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:47:50.677 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:47:50.679 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:47:50.679 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:47:51.128 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:ceph-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:47:51.311 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:librados-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 1/2 2020-05-24T13:47:51.622 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2/2 2020-05-24T13:47:51.675 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:ceph-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:47:51.676 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:51.676 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:47:51.677 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:51.677 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:51.677 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:47:51.736 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:librados-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 1/2 2020-05-24T13:47:51.790 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install libcephfs2 2020-05-24T13:47:52.026 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:52.071 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:52.073 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:52.073 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:52.073 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:52.074 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:52.239 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2/2 2020-05-24T13:47:52.239 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:52.239 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:52.240 INFO:teuthology.orchestra.run.smithi160.stdout:Dependency Installed: 2020-05-24T13:47:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:52.240 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:52.380 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install librados2 2020-05-24T13:47:52.674 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:52.719 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:52.720 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:52.721 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:52.721 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:52.721 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:52.738 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:53.083 INFO:teuthology.orchestra.run.smithi050.stdout:Package 2:libcephfs2-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:53.084 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do 2020-05-24T13:47:53.145 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install libcephfs-devel 2020-05-24T13:47:53.354 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:53.357 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:53.398 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:53.399 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:53.400 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:53.400 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:53.401 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:53.698 INFO:teuthology.orchestra.run.smithi160.stdout:Package 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:53.698 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do 2020-05-24T13:47:53.760 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install librbd1 2020-05-24T13:47:54.002 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:54.050 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:54.051 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:54.051 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:54.051 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:54.051 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:54.068 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:54.416 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:47:54.417 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:54.417 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package libcephfs-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:54.418 INFO:teuthology.orchestra.run.smithi050.stdout:--> Processing Dependency: librados-devel = 2:14.2.9-537.g6fe217b2dd.el7 for package: 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2020-05-24T13:47:54.445 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:47:54.448 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package librados-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:54.706 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:54.846 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:55.040 INFO:teuthology.orchestra.run.smithi160.stdout:Package 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:55.041 INFO:teuthology.orchestra.run.smithi160.stdout:Nothing to do 2020-05-24T13:47:55.078 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:55.079 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:47:55.079 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:55.079 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:55.079 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository 2020-05-24T13:47:55.080 INFO:teuthology.orchestra.run.smithi050.stdout: Size 2020-05-24T13:47:55.080 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:55.080 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:47:55.080 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 17 k 2020-05-24T13:47:55.080 INFO:teuthology.orchestra.run.smithi050.stdout:Installing for dependencies: 2020-05-24T13:47:55.081 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 92 k 2020-05-24T13:47:55.081 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:55.081 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:47:55.081 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:47:55.081 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package (+1 Dependent package) 2020-05-24T13:47:55.082 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:55.082 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 109 k 2020-05-24T13:47:55.082 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 391 k 2020-05-24T13:47:55.082 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:47:55.102 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install python-ceph 2020-05-24T13:47:55.314 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:55.358 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:55.359 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:55.359 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:55.360 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:55.360 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:55.879 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2020-05-24T13:47:55.879 INFO:teuthology.orchestra.run.smithi050.stdout:Total 136 kB/s | 109 kB 00:00 2020-05-24T13:47:55.882 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:47:55.906 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:47:55.908 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:47:55.909 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:47:55.997 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:56.331 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:56.332 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:56.332 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package python-ceph-compat.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:47:56.474 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:librados-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 1/2 2020-05-24T13:47:56.704 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:47:56.727 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2/2 2020-05-24T13:47:56.866 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:librados-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 1/2 2020-05-24T13:47:56.891 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:56.892 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:47:56.893 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:56.893 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:56.893 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository 2020-05-24T13:47:56.894 INFO:teuthology.orchestra.run.smithi160.stdout: Size 2020-05-24T13:47:56.894 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:56.894 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:47:56.894 INFO:teuthology.orchestra.run.smithi160.stdout: python-ceph-compat x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 2.2 k 2020-05-24T13:47:56.895 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:56.895 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:47:56.895 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:47:56.895 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2020-05-24T13:47:56.895 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:56.895 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 2.2 k 2020-05-24T13:47:56.896 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 0 2020-05-24T13:47:56.896 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:47:57.392 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:libcephfs-devel-14.2.9-537.g6fe217b2dd.el7.x86_64 2/2 2020-05-24T13:47:57.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:57.393 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:47:57.393 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:57.393 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:57.393 INFO:teuthology.orchestra.run.smithi050.stdout:Dependency Installed: 2020-05-24T13:47:57.394 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:57.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:47:57.394 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:47:57.426 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:47:57.427 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:47:57.429 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:47:57.429 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:47:57.522 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install librados2 2020-05-24T13:47:57.788 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:57.835 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:57.836 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:57.836 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:57.837 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:57.837 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:58.050 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:python-ceph-compat-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:47:58.514 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:58.567 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:python-ceph-compat-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:47:58.568 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:58.568 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:47:58.568 INFO:teuthology.orchestra.run.smithi160.stdout: python-ceph-compat.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:47:58.569 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:47:58.569 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:47:58.686 INFO:teuthology.orchestra.run.smithi160:> sudo yum -y install rbd-fuse 2020-05-24T13:47:58.862 INFO:teuthology.orchestra.run.smithi050.stdout:Package 2:librados2-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:47:58.862 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do 2020-05-24T13:47:58.923 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install librbd1 2020-05-24T13:47:58.939 INFO:teuthology.orchestra.run.smithi160.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:58.982 INFO:teuthology.orchestra.run.smithi160.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:58.983 INFO:teuthology.orchestra.run.smithi160.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:58.983 INFO:teuthology.orchestra.run.smithi160.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-05-24T13:47:58.984 INFO:teuthology.orchestra.run.smithi160.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:58.984 INFO:teuthology.orchestra.run.smithi160.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:59.091 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:47:59.128 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:47:59.128 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:47:59.129 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:47:59.129 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:47:59.130 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:47:59.626 INFO:teuthology.orchestra.run.smithi160.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:59.750 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:47:59.963 INFO:teuthology.orchestra.run.smithi160.stdout:Resolving Dependencies 2020-05-24T13:47:59.964 INFO:teuthology.orchestra.run.smithi160.stdout:--> Running transaction check 2020-05-24T13:47:59.964 INFO:teuthology.orchestra.run.smithi160.stdout:---> Package rbd-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:00.088 INFO:teuthology.orchestra.run.smithi050.stdout:Package 2:librbd1-14.2.9-537.g6fe217b2dd.el7.x86_64 already installed and latest version 2020-05-24T13:48:00.088 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do 2020-05-24T13:48:00.152 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install python-ceph 2020-05-24T13:48:00.319 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:00.345 INFO:teuthology.orchestra.run.smithi160.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:00.356 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:00.357 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:00.357 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:48:00.358 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:00.358 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:00.531 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:00.532 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies Resolved 2020-05-24T13:48:00.532 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:00.533 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:48:00.533 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository 2020-05-24T13:48:00.533 INFO:teuthology.orchestra.run.smithi160.stdout: Size 2020-05-24T13:48:00.533 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:48:00.533 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2020-05-24T13:48:00.534 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 75 k 2020-05-24T13:48:00.534 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:00.534 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2020-05-24T13:48:00.534 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2020-05-24T13:48:00.535 INFO:teuthology.orchestra.run.smithi160.stdout:Install 1 Package 2020-05-24T13:48:00.535 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:00.535 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 75 k 2020-05-24T13:48:00.535 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 196 k 2020-05-24T13:48:00.535 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading packages: 2020-05-24T13:48:00.961 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2020-05-24T13:48:00.982 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:00.984 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2020-05-24T13:48:00.986 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded 2020-05-24T13:48:00.987 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2020-05-24T13:48:01.325 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:48:01.326 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:48:01.326 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package python-ceph-compat.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:01.663 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : 2:rbd-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:01.720 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:01.916 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:01.916 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:48:01.917 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:01.917 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:48:01.917 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository 2020-05-24T13:48:01.918 INFO:teuthology.orchestra.run.smithi050.stdout: Size 2020-05-24T13:48:01.918 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:48:01.918 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:48:01.918 INFO:teuthology.orchestra.run.smithi050.stdout: python-ceph-compat x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 2.2 k 2020-05-24T13:48:01.918 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:01.919 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:48:01.919 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:48:01.919 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2020-05-24T13:48:01.919 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:01.919 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 2.2 k 2020-05-24T13:48:01.920 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 0 2020-05-24T13:48:01.920 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:48:02.071 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : 2:rbd-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:02.071 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:02.071 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2020-05-24T13:48:02.072 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:02.072 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:02.072 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2020-05-24T13:48:02.196 DEBUG:teuthology.parallel:result is None 2020-05-24T13:48:02.288 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:48:02.289 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:48:02.291 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:48:02.292 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:48:02.929 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:python-ceph-compat-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:03.388 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:python-ceph-compat-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:03.388 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:03.388 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:48:03.389 INFO:teuthology.orchestra.run.smithi050.stdout: python-ceph-compat.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:03.389 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:03.389 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:48:03.511 INFO:teuthology.orchestra.run.smithi050:> sudo yum -y install rbd-fuse 2020-05-24T13:48:03.750 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-05-24T13:48:03.794 INFO:teuthology.orchestra.run.smithi050.stdout:Loading mirror speeds from cached hostfile 2020-05-24T13:48:03.795 INFO:teuthology.orchestra.run.smithi050.stdout: * base: mirrors.mit.edu 2020-05-24T13:48:03.796 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: ftp.linux.ncsu.edu 2020-05-24T13:48:03.796 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: mirror.facebook.net 2020-05-24T13:48:03.796 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.vcu.edu 2020-05-24T13:48:04.448 INFO:teuthology.orchestra.run.smithi050.stdout:7 packages excluded due to repository priority protections 2020-05-24T13:48:04.792 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-05-24T13:48:04.792 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-05-24T13:48:04.793 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package rbd-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 will be installed 2020-05-24T13:48:05.193 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-05-24T13:48:05.386 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:05.387 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-05-24T13:48:05.388 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:05.389 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:48:05.389 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository 2020-05-24T13:48:05.389 INFO:teuthology.orchestra.run.smithi050.stdout: Size 2020-05-24T13:48:05.390 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:48:05.390 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-05-24T13:48:05.390 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:14.2.9-537.g6fe217b2dd.el7 ceph 75 k 2020-05-24T13:48:05.391 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-05-24T13:48:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-05-24T13:48:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2020-05-24T13:48:05.392 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:05.392 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 75 k 2020-05-24T13:48:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 196 k 2020-05-24T13:48:05.393 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-05-24T13:48:05.840 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-05-24T13:48:05.867 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-05-24T13:48:05.867 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-05-24T13:48:05.868 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-05-24T13:48:06.535 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : 2:rbd-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:07.051 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : 2:rbd-fuse-14.2.9-537.g6fe217b2dd.el7.x86_64 1/1 2020-05-24T13:48:07.052 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:07.052 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2020-05-24T13:48:07.052 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse.x86_64 2:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:07.052 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:07.053 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2020-05-24T13:48:07.174 DEBUG:teuthology.parallel:result is None 2020-05-24T13:48:07.174 INFO:teuthology.orchestra.run.smithi050:> sudo lsb_release -is 2020-05-24T13:48:07.313 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS 2020-05-24T13:48:07.316 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:48:07.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:48:07.564 INFO:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-05-24T13:48:07.705 INFO:teuthology.orchestra.run.smithi050.stdout:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:07.706 INFO:teuthology.packaging:The installed version of ceph is 14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:07.706 INFO:teuthology.task.install:The correct ceph version 14.2.9-537.g6fe217b2dd is installed. 2020-05-24T13:48:07.707 INFO:teuthology.orchestra.run.smithi160:> sudo lsb_release -is 2020-05-24T13:48:07.856 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS 2020-05-24T13:48:07.858 DEBUG:teuthology.misc:System to be installed: CentOS 2020-05-24T13:48:07.859 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=6fe217b2dddadf1baae6a015a57f83f4283a6078 2020-05-24T13:48:08.115 INFO:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-05-24T13:48:08.245 INFO:teuthology.orchestra.run.smithi160.stdout:14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:08.245 INFO:teuthology.packaging:The installed version of ceph is 14.2.9-537.g6fe217b2dd.el7 2020-05-24T13:48:08.246 INFO:teuthology.task.install:The correct ceph version 14.2.9-537.g6fe217b2dd is installed. 2020-05-24T13:48:08.246 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-05-24T13:48:08.247 INFO:teuthology.orchestra.run.smithi050:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-05-24T13:48:08.367 INFO:teuthology.orchestra.run.smithi160:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-05-24T13:48:08.485 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-05-24T13:48:08.486 INFO:teuthology.orchestra.run.smithi050:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-05-24T13:48:08.614 INFO:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-05-24T13:48:08.722 INFO:teuthology.orchestra.run.smithi160:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-05-24T13:48:08.842 INFO:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-05-24T13:48:08.946 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-05-24T13:48:08.947 INFO:teuthology.orchestra.run.smithi050:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-05-24T13:48:09.067 INFO:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-05-24T13:48:09.178 INFO:teuthology.orchestra.run.smithi160:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-05-24T13:48:09.299 INFO:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-05-24T13:48:09.406 INFO:teuthology.run_tasks:Running task ceph... 2020-05-24T13:48:09.480 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-05-24T13:48:09.481 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 777 /var/log/ceph 2020-05-24T13:48:09.486 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 777 /var/log/ceph 2020-05-24T13:48:09.602 INFO:tasks.ceph:Disabling ceph logrotate... 2020-05-24T13:48:09.602 INFO:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/logrotate.d/ceph 2020-05-24T13:48:09.606 INFO:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/logrotate.d/ceph 2020-05-24T13:48:09.717 INFO:tasks.ceph:Creating extra log directories... 2020-05-24T13:48:09.717 INFO:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-05-24T13:48:09.721 INFO:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-05-24T13:48:09.827 INFO:tasks.ceph:Creating ceph cluster c1... 2020-05-24T13:48:09.827 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'c1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-05-24T13:48:09.828 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087414', 'branch': 'nautilus', 'description': 'rgw/multisite/{clusters.yaml frontend/civetweb.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml}', 'email': 'tchaikov@gmail.com', 'first_in_suite': False, 'job_id': '5087414', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi', 'nuke-on-error': True, 'os_type': 'centos', 'overrides': {'admin_socket': {'branch': 'nautilus'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078'}}, 'rgw': {'compression type': 'random', 'frontend': 'civetweb'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}, {'endpoints': ['c1.client.1'], 'name': 'test-zone3'}, {'endpoints': ['c2.client.1'], 'is_pubsub': True, 'name': 'test-zone4'}]}]}, 'workunit': {'branch': 'nautilus-py3', 'sha1': '2e1d102f0cb696365106419cf1e42700ba50a8ff'}}, 'owner': 'scheduled_kchai@teuthology', 'priority': 90, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'], ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'nautilus-py3', 'suite_path': '/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/tchaikov/ceph.git', 'suite_sha1': '2e1d102f0cb696365106419cf1e42700ba50a8ff', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}}, {'ceph': {'cluster': 'c2'}}, {'rgw': {'c1.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c1.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 60}}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.30709', 'targets': {'smithi050.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0YXVafo/xd3CdpgUWNATDXI81GKJ0ZICl+CwvHbakJEQbr4WNtjRDwwJfd6XQoW6raB3VIgqfoaPfxPNVsKHwentf6qQ3k3a9713qUVRkm/qJ0op2SHVSl1j6I9kpB66p1Qjlun9XbSOprdByFbBEIgbcSGw1S2W4YKJIJnAYhSR/gQZQJPqH2giCCJv34D63vfh4BvExEygU8Y0IITtnqPwLFugXEFfLAeCgZ7/LDiWtpaHOTbVgfiO320qWw5dx7l2CEiqxmJdjJrcS67+2TQY2kicRH8qWiHhWh3JueRZldMXP1X3P7eqPY75SK8M5uYY8Guc5frCm1TcMSMlH', 'smithi160.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbjOIgGz2r4F2rEiE18SEix1zO1uILNCLKpnyWJ3HF/upvHd69PmvXN8b1Mw3h417K/U2qzhX+yBYW9GZpuDcSi6fAoI2ripRRrYZX4yj3NyYuflpwlfxCIR/WHOH3Q5hdHfRy3L9GeA6/M7uTsHu8lFv1+bzh1pkekIr97BNdu8SkZO4Jg/VCyA+iGX/Y/YkxSJkJ1ZeE2FQgXslTKRNRBzvCoheXa1nvETZL0EdJr1PqP8RQwpK5LCnuMutlWJYU8nvf4wTLhcv/7TpJyohyK8CCAMqMRSBUAt6jho5DfLfPm/HeCwzOPkZp3/qAr5cQCNujg7WMmgl2zl556cX3'}} 2020-05-24T13:48:09.829 INFO:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2020-05-24T13:48:09.833 INFO:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2020-05-24T13:48:09.904 INFO:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/run/ceph 2020-05-24T13:48:09.908 INFO:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/run/ceph 2020-05-24T13:48:10.064 DEBUG:teuthology.orchestra.remote:smithi050:/scratch_devs is 72B 2020-05-24T13:48:10.124 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-05-24T13:48:10.125 INFO:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-05-24T13:48:10.181 INFO:teuthology.orchestra.run.smithi050.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-0’ 2020-05-24T13:48:10.182 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:48:10.182 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 109898 Links: 1 2020-05-24T13:48:10.182 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:48:10.182 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:48:10.182 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2020-05-24 13:47:08.041171248 +0000 2020-05-24T13:48:10.183 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2020-05-24 13:45:25.591589546 +0000 2020-05-24T13:48:10.183 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2020-05-24 13:45:25.591589546 +0000 2020-05-24T13:48:10.183 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2020-05-24T13:48:10.209 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2020-05-24T13:48:10.210 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2020-05-24T13:48:10.210 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes (512 B) copied, 0.000239134 s, 2.1 MB/s 2020-05-24T13:48:10.215 INFO:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-05-24T13:48:10.269 INFO:teuthology.orchestra.run.smithi050.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-1’ 2020-05-24T13:48:10.270 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:48:10.270 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 94014 Links: 1 2020-05-24T13:48:10.270 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:48:10.271 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:48:10.271 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2020-05-24 13:47:08.041171248 +0000 2020-05-24T13:48:10.271 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2020-05-24 13:45:26.108576669 +0000 2020-05-24T13:48:10.271 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2020-05-24 13:45:26.108576669 +0000 2020-05-24T13:48:10.271 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2020-05-24T13:48:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2020-05-24T13:48:10.293 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2020-05-24T13:48:10.293 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes (512 B) copied, 0.000218212 s, 2.3 MB/s 2020-05-24T13:48:10.297 INFO:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-05-24T13:48:10.350 INFO:teuthology.orchestra.run.smithi050.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2020-05-24T13:48:10.350 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:48:10.350 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 94059 Links: 1 2020-05-24T13:48:10.351 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:48:10.351 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:48:10.351 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2020-05-24 13:47:08.041171248 +0000 2020-05-24T13:48:10.351 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2020-05-24 13:45:26.626563767 +0000 2020-05-24T13:48:10.351 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2020-05-24 13:45:26.626563767 +0000 2020-05-24T13:48:10.351 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2020-05-24T13:48:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2020-05-24T13:48:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2020-05-24T13:48:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes (512 B) copied, 0.000260233 s, 2.0 MB/s 2020-05-24T13:48:10.374 INFO:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-05-24T13:48:10.428 INFO:teuthology.orchestra.run.smithi050.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-3’ 2020-05-24T13:48:10.428 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:48:10.429 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 94104 Links: 1 2020-05-24T13:48:10.429 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:48:10.429 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:48:10.429 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2020-05-24 13:47:08.041171248 +0000 2020-05-24T13:48:10.429 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2020-05-24 13:45:27.099551986 +0000 2020-05-24T13:48:10.430 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2020-05-24 13:45:27.099551986 +0000 2020-05-24T13:48:10.430 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: - 2020-05-24T13:48:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2020-05-24T13:48:10.453 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2020-05-24T13:48:10.453 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes (512 B) copied, 0.000194571 s, 2.6 MB/s 2020-05-24T13:48:10.453 INFO:tasks.ceph:osd dev map: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:48:10.454 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}} 2020-05-24T13:48:10.454 INFO:tasks.ceph:Generating config... 2020-05-24T13:48:10.456 INFO:tasks.ceph:[client] debug rgw = 20 2020-05-24T13:48:10.456 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2020-05-24T13:48:10.456 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:48:10.457 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2020-05-24T13:48:10.457 INFO:tasks.ceph:[client] rgw data log num shards = 4 2020-05-24T13:48:10.457 INFO:tasks.ceph:[client] rgw md log max shards = 4 2020-05-24T13:48:10.458 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2020-05-24T13:48:10.458 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2020-05-24T13:48:10.458 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-05-24T13:48:10.458 INFO:tasks.ceph:[mgr] debug ms = 1 2020-05-24T13:48:10.459 INFO:tasks.ceph:[mon] debug mon = 20 2020-05-24T13:48:10.459 INFO:tasks.ceph:[mon] debug ms = 1 2020-05-24T13:48:10.459 INFO:tasks.ceph:[mon] debug paxos = 20 2020-05-24T13:48:10.460 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2020-05-24T13:48:10.460 INFO:tasks.ceph:[osd] debug filestore = 20 2020-05-24T13:48:10.460 INFO:tasks.ceph:[osd] debug journal = 20 2020-05-24T13:48:10.470 INFO:tasks.ceph:[osd] debug ms = 20 2020-05-24T13:48:10.470 INFO:tasks.ceph:[osd] debug osd = 25 2020-05-24T13:48:10.471 INFO:tasks.ceph:[osd] osd fast shutdown = False 2020-05-24T13:48:10.471 INFO:tasks.ceph:[osd.0] osd_max_omap_entries_per_request = 10 2020-05-24T13:48:10.471 INFO:tasks.ceph:[osd.1] osd_max_omap_entries_per_request = 1000 2020-05-24T13:48:10.471 INFO:tasks.ceph:[osd.2] osd_max_omap_entries_per_request = 10000 2020-05-24T13:48:10.472 INFO:tasks.ceph:Setting up c1.mon.a... 2020-05-24T13:48:10.473 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c1.keyring 2020-05-24T13:48:10.658 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/c1.keyring 2020-05-24T13:48:10.663 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c1.keyring 2020-05-24T13:48:10.828 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/c1.keyring 2020-05-24T13:48:10.948 DEBUG:tasks.ceph:Ceph mon addresses: [('c1.mon.a', '172.21.15.50')] 2020-05-24T13:48:10.949 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.50 --print /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:11.107 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:11.107 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: generated fsid c458a90b-5841-48fd-9e05-3dd82e2be174 2020-05-24T13:48:11.108 INFO:teuthology.orchestra.run.smithi050.stdout:epoch 0 2020-05-24T13:48:11.108 INFO:teuthology.orchestra.run.smithi050.stdout:fsid c458a90b-5841-48fd-9e05-3dd82e2be174 2020-05-24T13:48:11.108 INFO:teuthology.orchestra.run.smithi050.stdout:last_changed 2020-05-24 13:48:11.108452 2020-05-24T13:48:11.108 INFO:teuthology.orchestra.run.smithi050.stdout:created 2020-05-24 13:48:11.108452 2020-05-24T13:48:11.108 INFO:teuthology.orchestra.run.smithi050.stdout:min_mon_release 0 (unknown) 2020-05-24T13:48:11.109 INFO:teuthology.orchestra.run.smithi050.stdout:0: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.a 2020-05-24T13:48:11.109 INFO:teuthology.orchestra.run.smithi050.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c1.monmap (1 monitors) 2020-05-24T13:48:11.110 INFO:tasks.ceph:Writing /etc/ceph/c1.conf for FSID c458a90b-5841-48fd-9e05-3dd82e2be174... 2020-05-24T13:48:11.112 INFO:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2020-05-24T13:48:11.116 INFO:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2020-05-24T13:48:11.264 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2020-05-24T13:48:11.265 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2020-05-24T13:48:11.265 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-24T13:48:11.265 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2020-05-24T13:48:11.265 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.265 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2020-05-24T13:48:11.266 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.266 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2020-05-24T13:48:11.266 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.266 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2020-05-24T13:48:11.266 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2020-05-24T13:48:11.267 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.267 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2020-05-24T13:48:11.267 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2020-05-24T13:48:11.267 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.267 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-24T13:48:11.267 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2020-05-24T13:48:11.268 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.268 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2020-05-24T13:48:11.268 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.268 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2020-05-24T13:48:11.268 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2020-05-24T13:48:11.269 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2020-05-24T13:48:11.269 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2020-05-24T13:48:11.269 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2020-05-24T13:48:11.269 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2020-05-24T13:48:11.269 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2020-05-24T13:48:11.271 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-24T13:48:11.271 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-24T13:48:11.271 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.271 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-24T13:48:11.271 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.272 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2020-05-24T13:48:11.272 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.272 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2020-05-24T13:48:11.272 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.272 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2020-05-24T13:48:11.273 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2020-05-24T13:48:11.273 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.273 INFO:teuthology.orchestra.run.smithi050.stdout: # we see this fail in qa on *nautilus*; bump up retries 2020-05-24T13:48:11.273 INFO:teuthology.orchestra.run.smithi050.stdout: mon_client_directed_command_retry = 4 2020-05-24T13:48:11.274 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = 172.21.15.50 2020-05-24T13:48:11.274 INFO:teuthology.orchestra.run.smithi050.stdout: osd heartbeat grace = 40 2020-05-24T13:48:11.274 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = c458a90b-5841-48fd-9e05-3dd82e2be174 2020-05-24T13:48:11.274 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.274 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2020-05-24T13:48:11.275 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2020-05-24T13:48:11.275 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.275 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2020-05-24T13:48:11.275 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 600 2020-05-24T13:48:11.275 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.275 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2020-05-24T13:48:11.276 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2020-05-24T13:48:11.276 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.276 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2020-05-24T13:48:11.276 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2020-05-24T13:48:11.276 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2020-05-24T13:48:11.277 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.277 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2020-05-24T13:48:11.277 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2020-05-24T13:48:11.277 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.277 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2020-05-24T13:48:11.277 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.278 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2020-05-24T13:48:11.278 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.278 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2020-05-24T13:48:11.278 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.278 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2020-05-24T13:48:11.279 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2020-05-24T13:48:11.279 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.279 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2020-05-24T13:48:11.279 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2020-05-24T13:48:11.279 INFO:teuthology.orchestra.run.smithi050.stdout: debug filestore = 20 2020-05-24T13:48:11.280 INFO:teuthology.orchestra.run.smithi050.stdout: debug journal = 20 2020-05-24T13:48:11.280 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 20 2020-05-24T13:48:11.280 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 25 2020-05-24T13:48:11.280 INFO:teuthology.orchestra.run.smithi050.stdout: osd fast shutdown = False 2020-05-24T13:48:11.282 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.282 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2020-05-24T13:48:11.283 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2020-05-24T13:48:11.283 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2020-05-24T13:48:11.283 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2020-05-24T13:48:11.283 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2020-05-24T13:48:11.283 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:48:11.284 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:48:11.284 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.284 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2020-05-24T13:48:11.284 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2020-05-24T13:48:11.285 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2020-05-24T13:48:11.285 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2020-05-24T13:48:11.286 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2020-05-24T13:48:11.286 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2020-05-24T13:48:11.287 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2020-05-24T13:48:11.287 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:48:11.287 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2020-05-24T13:48:11.287 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2020-05-24T13:48:11.287 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:48:11.288 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd crush smoke test = False 2020-05-24T13:48:11.288 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:48:11.288 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2020-05-24T13:48:11.288 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2020-05-24T13:48:11.288 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2020-05-24T13:48:11.289 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2020-05-24T13:48:11.289 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-24T13:48:11.289 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-24T13:48:11.289 INFO:teuthology.orchestra.run.smithi050.stdout: debug rgw = 20 2020-05-24T13:48:11.289 INFO:teuthology.orchestra.run.smithi050.stdout: rgw crypt require ssl = False 2020-05-24T13:48:11.289 INFO:teuthology.orchestra.run.smithi050.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:48:11.290 INFO:teuthology.orchestra.run.smithi050.stdout: rgw curl low speed time = 300 2020-05-24T13:48:11.290 INFO:teuthology.orchestra.run.smithi050.stdout: rgw data log num shards = 4 2020-05-24T13:48:11.290 INFO:teuthology.orchestra.run.smithi050.stdout: rgw md log max shards = 4 2020-05-24T13:48:11.290 INFO:teuthology.orchestra.run.smithi050.stdout: rgw sync log trim interval = 0 2020-05-24T13:48:11.290 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2020-05-24T13:48:11.291 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.0] 2020-05-24T13:48:11.291 INFO:teuthology.orchestra.run.smithi050.stdout: osd_max_omap_entries_per_request = 10 2020-05-24T13:48:11.291 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.1] 2020-05-24T13:48:11.291 INFO:teuthology.orchestra.run.smithi050.stdout: osd_max_omap_entries_per_request = 1000 2020-05-24T13:48:11.291 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.2] 2020-05-24T13:48:11.292 INFO:teuthology.orchestra.run.smithi050.stdout: osd_max_omap_entries_per_request = 10000 2020-05-24T13:48:11.293 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2020-05-24T13:48:11.293 INFO:teuthology.orchestra.run.smithi160.stdout: chdir = "" 2020-05-24T13:48:11.293 INFO:teuthology.orchestra.run.smithi160.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-24T13:48:11.294 INFO:teuthology.orchestra.run.smithi160.stdout: auth supported = cephx 2020-05-24T13:48:11.294 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.294 INFO:teuthology.orchestra.run.smithi160.stdout: filestore xattr use omap = true 2020-05-24T13:48:11.294 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.294 INFO:teuthology.orchestra.run.smithi160.stdout: mon clock drift allowed = 1.000 2020-05-24T13:48:11.295 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.295 INFO:teuthology.orchestra.run.smithi160.stdout: osd crush chooseleaf type = 0 2020-05-24T13:48:11.295 INFO:teuthology.orchestra.run.smithi160.stdout: auth debug = true 2020-05-24T13:48:11.295 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.295 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on old message = true 2020-05-24T13:48:11.296 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on bug = true 2020-05-24T13:48:11.296 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-24T13:48:11.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon pg warn max object skew = 0 2020-05-24T13:48:11.296 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.296 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default size = 2 2020-05-24T13:48:11.297 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.297 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow primary affinity = true 2020-05-24T13:48:11.297 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow pg remap = true 2020-05-24T13:48:11.297 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on legacy crush tunables = false 2020-05-24T13:48:11.297 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on crush straw calc version zero = false 2020-05-24T13:48:11.298 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on no sortbitwise = false 2020-05-24T13:48:11.298 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on osd down out interval zero = false 2020-05-24T13:48:11.298 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on too few osds = false 2020-05-24T13:48:11.298 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-24T13:48:11.298 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-24T13:48:11.299 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.299 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-24T13:48:11.299 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.299 INFO:teuthology.orchestra.run.smithi160.stdout: osd default data pool replay window = 5 2020-05-24T13:48:11.299 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.299 INFO:teuthology.orchestra.run.smithi160.stdout: mon allow pool delete = true 2020-05-24T13:48:11.300 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.300 INFO:teuthology.orchestra.run.smithi160.stdout: mon cluster log file level = debug 2020-05-24T13:48:11.300 INFO:teuthology.orchestra.run.smithi160.stdout: debug asserts on shutdown = true 2020-05-24T13:48:11.300 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.301 INFO:teuthology.orchestra.run.smithi160.stdout: # we see this fail in qa on *nautilus*; bump up retries 2020-05-24T13:48:11.301 INFO:teuthology.orchestra.run.smithi160.stdout: mon_client_directed_command_retry = 4 2020-05-24T13:48:11.302 INFO:teuthology.orchestra.run.smithi160.stdout: mon host = 172.21.15.50 2020-05-24T13:48:11.302 INFO:teuthology.orchestra.run.smithi160.stdout: osd heartbeat grace = 40 2020-05-24T13:48:11.302 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = c458a90b-5841-48fd-9e05-3dd82e2be174 2020-05-24T13:48:11.302 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.302 INFO:teuthology.orchestra.run.smithi160.stdout:[osd] 2020-05-24T13:48:11.302 INFO:teuthology.orchestra.run.smithi160.stdout: osd journal size = 100 2020-05-24T13:48:11.303 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.303 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub load threshold = 5.0 2020-05-24T13:48:11.303 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub max interval = 600 2020-05-24T13:48:11.303 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.303 INFO:teuthology.orchestra.run.smithi160.stdout: osd recover clone overlap = true 2020-05-24T13:48:11.304 INFO:teuthology.orchestra.run.smithi160.stdout: osd recovery max chunk = 1048576 2020-05-24T13:48:11.304 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.304 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug shutdown = true 2020-05-24T13:48:11.304 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug op order = true 2020-05-24T13:48:11.304 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug verify stray on activate = true 2020-05-24T13:48:11.305 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.305 INFO:teuthology.orchestra.run.smithi160.stdout: osd open classes on start = true 2020-05-24T13:48:11.305 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug pg log writeout = true 2020-05-24T13:48:11.305 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.305 INFO:teuthology.orchestra.run.smithi160.stdout: osd deep scrub update digest min age = 30 2020-05-24T13:48:11.305 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.306 INFO:teuthology.orchestra.run.smithi160.stdout: osd map max advance = 10 2020-05-24T13:48:11.306 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.306 INFO:teuthology.orchestra.run.smithi160.stdout: journal zero on create = true 2020-05-24T13:48:11.306 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.306 INFO:teuthology.orchestra.run.smithi160.stdout: filestore ondisk finisher threads = 3 2020-05-24T13:48:11.307 INFO:teuthology.orchestra.run.smithi160.stdout: filestore apply finisher threads = 3 2020-05-24T13:48:11.307 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.307 INFO:teuthology.orchestra.run.smithi160.stdout: bdev debug aio = true 2020-05-24T13:48:11.307 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug misdirected ops = true 2020-05-24T13:48:11.307 INFO:teuthology.orchestra.run.smithi160.stdout: debug filestore = 20 2020-05-24T13:48:11.308 INFO:teuthology.orchestra.run.smithi160.stdout: debug journal = 20 2020-05-24T13:48:11.308 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 20 2020-05-24T13:48:11.308 INFO:teuthology.orchestra.run.smithi160.stdout: debug osd = 25 2020-05-24T13:48:11.308 INFO:teuthology.orchestra.run.smithi160.stdout: osd fast shutdown = False 2020-05-24T13:48:11.308 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.308 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr] 2020-05-24T13:48:11.309 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2020-05-24T13:48:11.309 INFO:teuthology.orchestra.run.smithi160.stdout: debug mgr = 20 2020-05-24T13:48:11.309 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2020-05-24T13:48:11.309 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2020-05-24T13:48:11.309 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:48:11.310 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:48:11.310 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.310 INFO:teuthology.orchestra.run.smithi160.stdout:[mon] 2020-05-24T13:48:11.310 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2020-05-24T13:48:11.311 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2020-05-24T13:48:11.311 INFO:teuthology.orchestra.run.smithi160.stdout: debug paxos = 20 2020-05-24T13:48:11.311 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2020-05-24T13:48:11.311 INFO:teuthology.orchestra.run.smithi160.stdout: mon data avail warn = 5 2020-05-24T13:48:11.312 INFO:teuthology.orchestra.run.smithi160.stdout: mon mgr mkfs grace = 240 2020-05-24T13:48:11.312 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:48:11.312 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd reporter subtree level = osd 2020-05-24T13:48:11.312 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd prime pg temp = true 2020-05-24T13:48:11.312 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:48:11.313 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd crush smoke test = False 2020-05-24T13:48:11.313 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:48:11.313 INFO:teuthology.orchestra.run.smithi160.stdout:[client] 2020-05-24T13:48:11.313 INFO:teuthology.orchestra.run.smithi160.stdout: rgw cache enabled = true 2020-05-24T13:48:11.313 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable ops log = true 2020-05-24T13:48:11.314 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable usage log = true 2020-05-24T13:48:11.314 INFO:teuthology.orchestra.run.smithi160.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-24T13:48:11.314 INFO:teuthology.orchestra.run.smithi160.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-24T13:48:11.314 INFO:teuthology.orchestra.run.smithi160.stdout: debug rgw = 20 2020-05-24T13:48:11.315 INFO:teuthology.orchestra.run.smithi160.stdout: rgw crypt require ssl = False 2020-05-24T13:48:11.315 INFO:teuthology.orchestra.run.smithi160.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:48:11.315 INFO:teuthology.orchestra.run.smithi160.stdout: rgw curl low speed time = 300 2020-05-24T13:48:11.315 INFO:teuthology.orchestra.run.smithi160.stdout: rgw data log num shards = 4 2020-05-24T13:48:11.316 INFO:teuthology.orchestra.run.smithi160.stdout: rgw md log max shards = 4 2020-05-24T13:48:11.316 INFO:teuthology.orchestra.run.smithi160.stdout: rgw sync log trim interval = 0 2020-05-24T13:48:11.316 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.a] 2020-05-24T13:48:11.316 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.0] 2020-05-24T13:48:11.316 INFO:teuthology.orchestra.run.smithi160.stdout: osd_max_omap_entries_per_request = 10 2020-05-24T13:48:11.317 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.1] 2020-05-24T13:48:11.317 INFO:teuthology.orchestra.run.smithi160.stdout: osd_max_omap_entries_per_request = 1000 2020-05-24T13:48:11.317 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.2] 2020-05-24T13:48:11.317 INFO:teuthology.orchestra.run.smithi160.stdout: osd_max_omap_entries_per_request = 10000 2020-05-24T13:48:11.318 INFO:tasks.ceph:Creating admin key on c1.mon.a... 2020-05-24T13:48:11.318 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c1.keyring 2020-05-24T13:48:11.505 INFO:tasks.ceph:Copying monmap to all nodes... 2020-05-24T13:48:11.583 DEBUG:teuthology.orchestra.remote:smithi050:/etc/ceph/c1.keyring is 206B 2020-05-24T13:48:11.747 DEBUG:teuthology.orchestra.remote:smithi050:/home/ubuntu/cephtest/c1.monmap is 177B 2020-05-24T13:48:11.831 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:48:11.832 INFO:teuthology.orchestra.run.smithi050:> sudo sh -c 'cat > /etc/ceph/c1.keyring' && sudo chmod 0644 /etc/ceph/c1.keyring 2020-05-24T13:48:11.981 INFO:teuthology.orchestra.run.smithi050:> cat > /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:12.068 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:48:12.068 INFO:teuthology.orchestra.run.smithi160:> sudo sh -c 'cat > /etc/ceph/c1.keyring' && sudo chmod 0644 /etc/ceph/c1.keyring 2020-05-24T13:48:12.217 INFO:teuthology.orchestra.run.smithi160:> cat > /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:12.301 INFO:tasks.ceph:Setting up mon nodes... 2020-05-24T13:48:12.302 INFO:tasks.ceph:Setting up mgr nodes... 2020-05-24T13:48:12.303 INFO:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mgr/c1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c1-x/keyring 2020-05-24T13:48:12.523 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mgr/c1-x/keyring 2020-05-24T13:48:12.528 INFO:tasks.ceph:Setting up mds nodes... 2020-05-24T13:48:12.529 INFO:tasks.ceph_client:Setting up client nodes... 2020-05-24T13:48:12.530 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c1.client.0.keyring && sudo chmod 0644 /etc/ceph/c1.client.0.keyring 2020-05-24T13:48:12.680 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/c1.client.0.keyring 2020-05-24T13:48:12.707 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c1.client.1.keyring && sudo chmod 0644 /etc/ceph/c1.client.1.keyring 2020-05-24T13:48:12.852 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/c1.client.1.keyring 2020-05-24T13:48:12.880 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-05-24T13:48:12.880 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}} 2020-05-24T13:48:12.881 INFO:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/c1-0 2020-05-24T13:48:12.991 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:48:12.992 INFO:tasks.ceph:role: c1.osd.0 2020-05-24T13:48:12.992 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:48:12.993 INFO:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-05-24T13:48:13.121 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:48:13.122 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:48:13.122 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:48:13.122 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:48:13.122 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:48:13.123 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:48:13.123 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:48:13.123 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:48:13.123 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:48:13.124 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2020-05-24T13:48:13.125 INFO:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c1-0 2020-05-24T13:48:13.237 INFO:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-0 2020-05-24T13:48:13.327 INFO:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/c1-1 2020-05-24T13:48:13.398 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:48:13.399 INFO:tasks.ceph:role: c1.osd.1 2020-05-24T13:48:13.400 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:48:13.400 INFO:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-05-24T13:48:13.506 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:48:13.506 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:48:13.506 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:48:13.507 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:48:13.507 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:48:13.507 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:48:13.507 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:48:13.507 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:48:13.508 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:48:13.509 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2020-05-24T13:48:13.509 INFO:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c1-1 2020-05-24T13:48:13.640 INFO:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-1 2020-05-24T13:48:13.729 INFO:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/c1-2 2020-05-24T13:48:13.805 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:48:13.805 INFO:tasks.ceph:role: c1.osd.2 2020-05-24T13:48:13.806 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:48:13.806 INFO:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-05-24T13:48:13.923 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:48:13.924 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:48:13.924 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:48:13.924 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:48:13.925 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:48:13.925 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:48:13.925 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:48:13.926 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:48:13.926 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:48:13.927 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2020-05-24T13:48:13.928 INFO:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c1-2 2020-05-24T13:48:14.054 INFO:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-2 2020-05-24T13:48:14.155 INFO:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:14.338 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:14.339 7f7bed5929c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-0/keyring: can't open /var/lib/ceph/osd/c1-0/keyring: (2) No such file or directory 2020-05-24T13:48:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:14.339 7f7bed5929c0 -1 created new key in keyring /var/lib/ceph/osd/c1-0/keyring 2020-05-24T13:48:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:14.339 7f7bed5929c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2020-05-24T13:48:14.340 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:14.339 7f7bed5929c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2020-05-24T13:48:14.340 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:14.339 7f7bed5929c0 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2020-05-24T13:48:14.340 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:14.340 7f7bed5929c0 -1 bluestore(/var/lib/ceph/osd/c1-0) _read_fsid unparsable uuid 2020-05-24T13:48:16.389 INFO:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-0 2020-05-24T13:48:16.522 INFO:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:16.716 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:16.716 7fa2fb9b49c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-1/keyring: can't open /var/lib/ceph/osd/c1-1/keyring: (2) No such file or directory 2020-05-24T13:48:16.717 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:16.717 7fa2fb9b49c0 -1 created new key in keyring /var/lib/ceph/osd/c1-1/keyring 2020-05-24T13:48:16.717 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:16.717 7fa2fb9b49c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2020-05-24T13:48:16.718 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:16.717 7fa2fb9b49c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2020-05-24T13:48:16.718 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:16.717 7fa2fb9b49c0 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2020-05-24T13:48:16.719 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:16.718 7fa2fb9b49c0 -1 bluestore(/var/lib/ceph/osd/c1-1) _read_fsid unparsable uuid 2020-05-24T13:48:18.767 INFO:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-1 2020-05-24T13:48:18.887 INFO:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:19.080 7f7d960369c0 -1 auth: error reading file: /var/lib/ceph/osd/c1-2/keyring: can't open /var/lib/ceph/osd/c1-2/keyring: (2) No such file or directory 2020-05-24T13:48:19.080 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:19.081 7f7d960369c0 -1 created new key in keyring /var/lib/ceph/osd/c1-2/keyring 2020-05-24T13:48:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:19.081 7f7d960369c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2020-05-24T13:48:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:19.081 7f7d960369c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2020-05-24T13:48:19.081 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:19.081 7f7d960369c0 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2020-05-24T13:48:19.082 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:48:19.082 7f7d960369c0 -1 bluestore(/var/lib/ceph/osd/c1-2) _read_fsid unparsable uuid 2020-05-24T13:48:21.128 INFO:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-2 2020-05-24T13:48:21.250 INFO:tasks.ceph:Reading keys from all nodes... 2020-05-24T13:48:21.250 INFO:teuthology.orchestra.run.smithi050:> mktemp 2020-05-24T13:48:21.327 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.yMiV0xioea 2020-05-24T13:48:21.328 INFO:teuthology.orchestra.run.smithi050:> sudo cp /var/lib/ceph/mgr/c1-x/keyring /tmp/tmp.yMiV0xioea 2020-05-24T13:48:21.431 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.yMiV0xioea 2020-05-24T13:48:21.596 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.yMiV0xioea is 56B 2020-05-24T13:48:21.665 INFO:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.yMiV0xioea 2020-05-24T13:48:21.733 INFO:teuthology.orchestra.run.smithi050:> mktemp 2020-05-24T13:48:21.801 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.fIXM3Rv97y 2020-05-24T13:48:21.802 INFO:teuthology.orchestra.run.smithi050:> sudo cp /var/lib/ceph/osd/c1-0/keyring /tmp/tmp.fIXM3Rv97y 2020-05-24T13:48:21.895 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.fIXM3Rv97y 2020-05-24T13:48:22.058 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.fIXM3Rv97y is 56B 2020-05-24T13:48:22.138 INFO:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.fIXM3Rv97y 2020-05-24T13:48:22.214 INFO:teuthology.orchestra.run.smithi050:> mktemp 2020-05-24T13:48:22.285 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.3fHXkdVTt7 2020-05-24T13:48:22.286 INFO:teuthology.orchestra.run.smithi050:> sudo cp /var/lib/ceph/osd/c1-1/keyring /tmp/tmp.3fHXkdVTt7 2020-05-24T13:48:22.395 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.3fHXkdVTt7 2020-05-24T13:48:22.603 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.3fHXkdVTt7 is 56B 2020-05-24T13:48:22.692 INFO:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.3fHXkdVTt7 2020-05-24T13:48:22.764 INFO:teuthology.orchestra.run.smithi050:> mktemp 2020-05-24T13:48:22.831 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.5qyZv7GjyG 2020-05-24T13:48:22.832 INFO:teuthology.orchestra.run.smithi050:> sudo cp /var/lib/ceph/osd/c1-2/keyring /tmp/tmp.5qyZv7GjyG 2020-05-24T13:48:22.938 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.5qyZv7GjyG 2020-05-24T13:48:23.106 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.5qyZv7GjyG is 56B 2020-05-24T13:48:23.184 INFO:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.5qyZv7GjyG 2020-05-24T13:48:23.383 DEBUG:teuthology.orchestra.remote:smithi050:/etc/ceph/c1.client.0.keyring is 59B 2020-05-24T13:48:23.600 DEBUG:teuthology.orchestra.remote:smithi050:/etc/ceph/c1.client.1.keyring is 59B 2020-05-24T13:48:23.699 INFO:tasks.ceph:Adding keys to all mons... 2020-05-24T13:48:23.700 INFO:teuthology.orchestra.run.smithi050:> sudo tee -a /etc/ceph/c1.keyring 2020-05-24T13:48:23.808 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr.x] 2020-05-24T13:48:23.809 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQAce8pe2lM/HxAA0RuycRh+WwVkODqd0ZVpGg== 2020-05-24T13:48:23.809 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.0] 2020-05-24T13:48:23.809 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQAee8peiv9BFBAAR2ydUDX4Gzz97n0M1b39uA== 2020-05-24T13:48:23.809 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.1] 2020-05-24T13:48:23.810 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQAge8permXEKhAA0sikJiLW4AeoMak5MZCDBw== 2020-05-24T13:48:23.810 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.2] 2020-05-24T13:48:23.810 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQAje8pegO3aBBAAwJaL6YP7T/5m2XqPZzqVpA== 2020-05-24T13:48:23.810 INFO:teuthology.orchestra.run.smithi050.stdout:[client.0] 2020-05-24T13:48:23.810 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQAce8pe8majKBAAiJj1GmAFEQ7x05jemW1JTA== 2020-05-24T13:48:23.811 INFO:teuthology.orchestra.run.smithi050.stdout:[client.1] 2020-05-24T13:48:23.811 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQAce8peIbffMhAA7d5DhvVZmUsxB140aGVCnA== 2020-05-24T13:48:23.812 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-05-24T13:48:24.001 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:48:24.187 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:48:24.391 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:48:24.595 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-24T13:48:24.804 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-24T13:48:25.017 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-05-24T13:48:25.017 INFO:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mon/c1-a 2020-05-24T13:48:25.119 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c1 --mkfs -i a --monmap /home/ubuntu/cephtest/c1.monmap --keyring /etc/ceph/c1.keyring 2020-05-24T13:48:25.286 INFO:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c1-a 2020-05-24T13:48:25.392 INFO:teuthology.orchestra.run.smithi050:> rm -- /home/ubuntu/cephtest/c1.monmap 2020-05-24T13:48:25.478 INFO:tasks.ceph:Starting mon daemons in cluster c1... 2020-05-24T13:48:25.479 DEBUG:teuthology.misc:running c1.mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2020-05-24T13:48:25.479 INFO:tasks.ceph.c1.mon.a:Restarting daemon 2020-05-24T13:48:25.480 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.mon.a.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster c1 -i a 2020-05-24T13:48:25.485 INFO:tasks.ceph.c1.mon.a:Started 2020-05-24T13:48:25.485 INFO:tasks.ceph:Starting mgr daemons in cluster c1... 2020-05-24T13:48:25.486 INFO:tasks.ceph.c1.mgr.x:Restarting daemon 2020-05-24T13:48:25.487 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster c1 -i x 2020-05-24T13:48:25.490 INFO:tasks.ceph.c1.mgr.x:Started 2020-05-24T13:48:25.491 INFO:tasks.ceph:Setting crush tunables to default 2020-05-24T13:48:25.491 INFO:teuthology.orchestra.run.smithi050:> sudo ceph --cluster c1 osd crush tunables default 2020-05-24T13:48:31.246 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:00:05.292 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T13:48:37.050 INFO:teuthology.orchestra.run.smithi050.stderr:adjusted tunables profile to default 2020-05-24T13:48:37.074 INFO:tasks.ceph:Starting osd daemons in cluster c1... 2020-05-24T13:48:37.075 INFO:teuthology.orchestra.run.smithi050:> mktemp 2020-05-24T13:48:37.156 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.M46low9MqN 2020-05-24T13:48:37.156 INFO:teuthology.orchestra.run.smithi050:> sudo cp /var/lib/ceph/osd/c1-0/fsid /tmp/tmp.M46low9MqN 2020-05-24T13:48:37.240 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.M46low9MqN 2020-05-24T13:48:37.364 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.M46low9MqN is 37B 2020-05-24T13:48:37.412 INFO:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.M46low9MqN 2020-05-24T13:48:37.455 INFO:teuthology.orchestra.run.smithi050:> mktemp 2020-05-24T13:48:37.549 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.llC6uYztEA 2020-05-24T13:48:37.550 INFO:teuthology.orchestra.run.smithi050:> sudo cp /var/lib/ceph/osd/c1-1/fsid /tmp/tmp.llC6uYztEA 2020-05-24T13:48:37.609 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.llC6uYztEA 2020-05-24T13:48:37.707 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.llC6uYztEA is 37B 2020-05-24T13:48:37.754 INFO:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.llC6uYztEA 2020-05-24T13:48:37.796 INFO:teuthology.orchestra.run.smithi050:> mktemp 2020-05-24T13:48:37.891 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.kONYrSVZzJ 2020-05-24T13:48:37.892 INFO:teuthology.orchestra.run.smithi050:> sudo cp /var/lib/ceph/osd/c1-2/fsid /tmp/tmp.kONYrSVZzJ 2020-05-24T13:48:37.954 INFO:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.kONYrSVZzJ 2020-05-24T13:48:38.054 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.kONYrSVZzJ is 37B 2020-05-24T13:48:38.101 INFO:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.kONYrSVZzJ 2020-05-24T13:48:38.143 INFO:teuthology.orchestra.run.smithi050:> sudo ceph --cluster c1 osd new 8311e7e2-4cd3-4a3b-9da9-90379afcd9b4 0 2020-05-24T13:48:39.552 INFO:teuthology.orchestra.run.smithi050.stdout:0 2020-05-24T13:48:39.572 INFO:teuthology.orchestra.run.smithi050:> sudo ceph --cluster c1 osd new be5dc296-159c-4065-8aa2-fa73c15f91bf 1 2020-05-24T13:48:40.824 INFO:teuthology.orchestra.run.smithi050.stdout:1 2020-05-24T13:48:40.852 INFO:teuthology.orchestra.run.smithi050:> sudo ceph --cluster c1 osd new f09f00b4-e6cf-49ad-a53c-24bddd850d00 2 2020-05-24T13:48:41.912 INFO:teuthology.orchestra.run.smithi050.stdout:2 2020-05-24T13:48:41.936 DEBUG:teuthology.misc:running c1.osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:48:41.937 INFO:tasks.ceph.c1.osd.0:Restarting daemon 2020-05-24T13:48:41.937 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.osd.0.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c1 -i 0 2020-05-24T13:48:41.941 INFO:tasks.ceph.c1.osd.0:Started 2020-05-24T13:48:41.942 DEBUG:teuthology.misc:running c1.osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:48:41.942 INFO:tasks.ceph.c1.osd.1:Restarting daemon 2020-05-24T13:48:41.942 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.osd.1.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c1 -i 1 2020-05-24T13:48:41.945 INFO:tasks.ceph.c1.osd.1:Started 2020-05-24T13:48:41.945 DEBUG:teuthology.misc:running c1.osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:48:41.946 INFO:tasks.ceph.c1.osd.2:Restarting daemon 2020-05-24T13:48:41.946 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.osd.2.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c1 -i 2 2020-05-24T13:48:41.949 INFO:tasks.ceph.c1.osd.2:Started 2020-05-24T13:48:41.950 INFO:tasks.ceph:Waiting for OSDs to come up 2020-05-24T13:48:42.253 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2020-05-24T13:48:43.044 INFO:teuthology.misc.health.smithi050.stdout: 2020-05-24T13:48:43.044 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":5,"fsid":"c458a90b-5841-48fd-9e05-3dd82e2be174","created":"2020-05-24 13:48:33.778358","modified":"2020-05-24 13:48:41.815137","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"8311e7e2-4cd3-4a3b-9da9-90379afcd9b4","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"be5dc296-159c-4065-8aa2-fa73c15f91bf","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"f09f00b4-e6cf-49ad-a53c-24bddd850d00","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:48:43.063 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-05-24T13:48:48.509 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:==00:00:00:06.430 10501== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:48.566 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:==00:00:00:06.492 10501== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:49.164 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:==00:00:00:07.081 10502== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:49.187 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:2020-05-24 13:48:49.170 9ae5e40 -1 Falling back to public interface 2020-05-24T13:48:49.219 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:==00:00:00:07.125 10503== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:49.227 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:==00:00:00:07.144 10502== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:49.275 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:==00:00:00:07.181 10503== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:49.368 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2020-05-24T13:48:49.394 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:==00:00:00:07.320 10501== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:49.793 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:==00:00:00:07.718 10501== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:49.804 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:==00:00:00:07.729 10501== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:49.845 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:2020-05-24 13:48:49.831 9ae5e40 -1 Falling back to public interface 2020-05-24T13:48:49.874 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:2020-05-24 13:48:49.858 9ae5e40 -1 Falling back to public interface 2020-05-24T13:48:50.157 INFO:teuthology.misc.health.smithi050.stdout: 2020-05-24T13:48:50.157 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":5,"fsid":"c458a90b-5841-48fd-9e05-3dd82e2be174","created":"2020-05-24 13:48:33.778358","modified":"2020-05-24 13:48:41.815137","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"8311e7e2-4cd3-4a3b-9da9-90379afcd9b4","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"be5dc296-159c-4065-8aa2-fa73c15f91bf","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"f09f00b4-e6cf-49ad-a53c-24bddd850d00","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:48:50.170 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-05-24T13:48:50.175 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:==00:00:00:08.092 10502== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:50.212 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:==00:00:00:08.118 10503== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:50.576 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:==00:00:00:08.494 10502== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:50.591 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:==00:00:00:08.508 10502== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:50.625 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:==00:00:00:08.530 10503== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:50.634 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:==00:00:00:08.539 10503== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:50.859 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:==00:00:00:08.782 10501== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:51.670 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:==00:00:00:09.588 10502== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:51.707 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:==00:00:00:09.612 10503== Warning: unimplemented fcntl command: 1038 2020-05-24T13:48:52.336 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:2020-05-24 13:48:52.331 9ae5e40 -1 osd.0 0 log_to_monitors {default=true} 2020-05-24T13:48:53.001 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:2020-05-24 13:48:52.999 9ae5e40 -1 osd.1 0 log_to_monitors {default=true} 2020-05-24T13:48:53.098 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:2020-05-24 13:48:53.097 9ae5e40 -1 osd.2 0 log_to_monitors {default=true} 2020-05-24T13:48:54.360 INFO:tasks.ceph.c1.osd.0.smithi050.stderr:2020-05-24 13:48:54.306 290bd700 -1 osd.0 0 waiting for initial osdmap 2020-05-24T13:48:55.230 INFO:tasks.ceph.c1.osd.1.smithi050.stderr:2020-05-24 13:48:55.227 28cbc700 -1 osd.1 0 waiting for initial osdmap 2020-05-24T13:48:55.333 INFO:tasks.ceph.c1.osd.2.smithi050.stderr:2020-05-24 13:48:55.330 26cbb700 -1 osd.2 0 waiting for initial osdmap 2020-05-24T13:48:56.474 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2020-05-24T13:48:57.339 INFO:teuthology.misc.health.smithi050.stdout: 2020-05-24T13:48:57.339 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":9,"fsid":"c458a90b-5841-48fd-9e05-3dd82e2be174","created":"2020-05-24 13:48:33.778358","modified":"2020-05-24 13:48:55.985951","last_up_change":"2020-05-24 13:48:55.985951","last_in_change":"2020-05-24 13:48:55.985951","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"8311e7e2-4cd3-4a3b-9da9-90379afcd9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":10501},{"type":"v1","addr":"172.21.15.50:6803","nonce":10501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":10501},{"type":"v1","addr":"172.21.15.50:6805","nonce":10501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":10501},{"type":"v1","addr":"172.21.15.50:6809","nonce":10501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":10501},{"type":"v1","addr":"172.21.15.50:6807","nonce":10501}]},"public_addr":"172.21.15.50:6803/10501","cluster_addr":"172.21.15.50:6805/10501","heartbeat_back_addr":"172.21.15.50:6809/10501","heartbeat_front_addr":"172.21.15.50:6807/10501","state":["exists","up"]},{"osd":1,"uuid":"be5dc296-159c-4065-8aa2-fa73c15f91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":10502},{"type":"v1","addr":"172.21.15.50:6811","nonce":10502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":10502},{"type":"v1","addr":"172.21.15.50:6813","nonce":10502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":10502},{"type":"v1","addr":"172.21.15.50:6821","nonce":10502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6815","nonce":10502},{"type":"v1","addr":"172.21.15.50:6817","nonce":10502}]},"public_addr":"172.21.15.50:6811/10502","cluster_addr":"172.21.15.50:6813/10502","heartbeat_back_addr":"172.21.15.50:6821/10502","heartbeat_front_addr":"172.21.15.50:6817/10502","state":["exists","up"]},{"osd":2,"uuid":"f09f00b4-e6cf-49ad-a53c-24bddd850d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":10503},{"type":"v1","addr":"172.21.15.50:6816","nonce":10503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":10503},{"type":"v1","addr":"172.21.15.50:6820","nonce":10503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":10503},{"type":"v1","addr":"172.21.15.50:6825","nonce":10503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":10503},{"type":"v1","addr":"172.21.15.50:6823","nonce":10503}]},"public_addr":"172.21.15.50:6816/10503","cluster_addr":"172.21.15.50:6820/10503","heartbeat_back_addr":"172.21.15.50:6825/10503","heartbeat_front_addr":"172.21.15.50:6823/10503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:48:57.364 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-05-24T13:48:57.365 INFO:tasks.ceph:Creating RBD pool 2020-05-24T13:48:57.365 INFO:teuthology.orchestra.run.smithi050:> sudo ceph --cluster c1 osd pool create rbd 8 2020-05-24T13:48:59.427 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'rbd' created 2020-05-24T13:48:59.450 INFO:teuthology.orchestra.run.smithi050:> sudo ceph --cluster c1 osd pool application enable rbd rbd --yes-i-really-mean-it 2020-05-24T13:49:01.397 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rbd' on pool 'rbd' 2020-05-24T13:49:01.410 INFO:tasks.ceph:Starting mds daemons in cluster c1... 2020-05-24T13:49:01.411 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:02.291 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:02.292 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":12,"fsid":"c458a90b-5841-48fd-9e05-3dd82e2be174","created":"2020-05-24 13:48:33.778358","modified":"2020-05-24 13:49:01.306407","last_up_change":"2020-05-24 13:48:55.985951","last_in_change":"2020-05-24 13:48:55.985951","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:48:58.285559","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8311e7e2-4cd3-4a3b-9da9-90379afcd9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":10501},{"type":"v1","addr":"172.21.15.50:6803","nonce":10501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":10501},{"type":"v1","addr":"172.21.15.50:6805","nonce":10501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":10501},{"type":"v1","addr":"172.21.15.50:6809","nonce":10501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":10501},{"type":"v1","addr":"172.21.15.50:6807","nonce":10501}]},"public_addr":"172.21.15.50:6803/10501","cluster_addr":"172.21.15.50:6805/10501","heartbeat_back_addr":"172.21.15.50:6809/10501","heartbeat_front_addr":"172.21.15.50:6807/10501","state":["exists","up"]},{"osd":1,"uuid":"be5dc296-159c-4065-8aa2-fa73c15f91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":10502},{"type":"v1","addr":"172.21.15.50:6811","nonce":10502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":10502},{"type":"v1","addr":"172.21.15.50:6813","nonce":10502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":10502},{"type":"v1","addr":"172.21.15.50:6821","nonce":10502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6815","nonce":10502},{"type":"v1","addr":"172.21.15.50:6817","nonce":10502}]},"public_addr":"172.21.15.50:6811/10502","cluster_addr":"172.21.15.50:6813/10502","heartbeat_back_addr":"172.21.15.50:6821/10502","heartbeat_front_addr":"172.21.15.50:6817/10502","state":["exists","up"]},{"osd":2,"uuid":"f09f00b4-e6cf-49ad-a53c-24bddd850d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":10503},{"type":"v1","addr":"172.21.15.50:6816","nonce":10503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":10503},{"type":"v1","addr":"172.21.15.50:6820","nonce":10503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":10503},{"type":"v1","addr":"172.21.15.50:6825","nonce":10503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":10503},{"type":"v1","addr":"172.21.15.50:6823","nonce":10503}]},"public_addr":"172.21.15.50:6816/10503","cluster_addr":"172.21.15.50:6820/10503","heartbeat_back_addr":"172.21.15.50:6825/10503","heartbeat_front_addr":"172.21.15.50:6823/10503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:02.318 INFO:tasks.ceph.ceph_manager.c1:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2020-05-24 13:48:58.285559', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-05-24T13:49:02.319 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd pool get rbd pg_num 2020-05-24T13:49:03.251 INFO:teuthology.orchestra.run.smithi050.stdout:pg_num: 8 2020-05-24T13:49:03.267 INFO:tasks.ceph:Waiting until c1 daemons up and pgs clean... 2020-05-24T13:49:03.268 INFO:tasks.ceph.ceph_manager.c1:waiting for mgr available 2020-05-24T13:49:03.268 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 mgr dump --format=json 2020-05-24T13:49:04.359 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:04.376 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":9595},{"type":"v1","addr":"172.21.15.50:6801","nonce":9595}]},"active_addr":"172.21.15.50:6801/9595","active_change":"2020-05-24 13:48:38.492669","available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout_min":{"name":"inventory_cache_timeout_min","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"]}} 2020-05-24T13:49:04.378 INFO:tasks.ceph.ceph_manager.c1:mgr available! 2020-05-24T13:49:04.681 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:05.594 INFO:teuthology.misc.health.smithi050.stdout: 2020-05-24T13:49:05.594 INFO:teuthology.misc.health.smithi050.stdout:{"epoch":13,"fsid":"c458a90b-5841-48fd-9e05-3dd82e2be174","created":"2020-05-24 13:48:33.778358","modified":"2020-05-24 13:49:02.477427","last_up_change":"2020-05-24 13:48:55.985951","last_in_change":"2020-05-24 13:48:55.985951","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:48:58.285559","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8311e7e2-4cd3-4a3b-9da9-90379afcd9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":10501},{"type":"v1","addr":"172.21.15.50:6803","nonce":10501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":10501},{"type":"v1","addr":"172.21.15.50:6805","nonce":10501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":10501},{"type":"v1","addr":"172.21.15.50:6809","nonce":10501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":10501},{"type":"v1","addr":"172.21.15.50:6807","nonce":10501}]},"public_addr":"172.21.15.50:6803/10501","cluster_addr":"172.21.15.50:6805/10501","heartbeat_back_addr":"172.21.15.50:6809/10501","heartbeat_front_addr":"172.21.15.50:6807/10501","state":["exists","up"]},{"osd":1,"uuid":"be5dc296-159c-4065-8aa2-fa73c15f91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":10502},{"type":"v1","addr":"172.21.15.50:6811","nonce":10502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":10502},{"type":"v1","addr":"172.21.15.50:6813","nonce":10502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":10502},{"type":"v1","addr":"172.21.15.50:6821","nonce":10502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6815","nonce":10502},{"type":"v1","addr":"172.21.15.50:6817","nonce":10502}]},"public_addr":"172.21.15.50:6811/10502","cluster_addr":"172.21.15.50:6813/10502","heartbeat_back_addr":"172.21.15.50:6821/10502","heartbeat_front_addr":"172.21.15.50:6817/10502","state":["exists","up"]},{"osd":2,"uuid":"f09f00b4-e6cf-49ad-a53c-24bddd850d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":10503},{"type":"v1","addr":"172.21.15.50:6816","nonce":10503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":10503},{"type":"v1","addr":"172.21.15.50:6820","nonce":10503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":10503},{"type":"v1","addr":"172.21.15.50:6825","nonce":10503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":10503},{"type":"v1","addr":"172.21.15.50:6823","nonce":10503}]},"public_addr":"172.21.15.50:6816/10503","cluster_addr":"172.21.15.50:6820/10503","heartbeat_back_addr":"172.21.15.50:6825/10503","heartbeat_front_addr":"172.21.15.50:6823/10503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:05.613 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-05-24T13:49:05.614 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd dump --format=json 2020-05-24T13:49:06.575 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:06.576 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":13,"fsid":"c458a90b-5841-48fd-9e05-3dd82e2be174","created":"2020-05-24 13:48:33.778358","modified":"2020-05-24 13:49:02.477427","last_up_change":"2020-05-24 13:48:55.985951","last_in_change":"2020-05-24 13:48:55.985951","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:48:58.285559","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"8311e7e2-4cd3-4a3b-9da9-90379afcd9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":10501},{"type":"v1","addr":"172.21.15.50:6803","nonce":10501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":10501},{"type":"v1","addr":"172.21.15.50:6805","nonce":10501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":10501},{"type":"v1","addr":"172.21.15.50:6809","nonce":10501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":10501},{"type":"v1","addr":"172.21.15.50:6807","nonce":10501}]},"public_addr":"172.21.15.50:6803/10501","cluster_addr":"172.21.15.50:6805/10501","heartbeat_back_addr":"172.21.15.50:6809/10501","heartbeat_front_addr":"172.21.15.50:6807/10501","state":["exists","up"]},{"osd":1,"uuid":"be5dc296-159c-4065-8aa2-fa73c15f91bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":10502},{"type":"v1","addr":"172.21.15.50:6811","nonce":10502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":10502},{"type":"v1","addr":"172.21.15.50:6813","nonce":10502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":10502},{"type":"v1","addr":"172.21.15.50:6821","nonce":10502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6815","nonce":10502},{"type":"v1","addr":"172.21.15.50:6817","nonce":10502}]},"public_addr":"172.21.15.50:6811/10502","cluster_addr":"172.21.15.50:6813/10502","heartbeat_back_addr":"172.21.15.50:6821/10502","heartbeat_front_addr":"172.21.15.50:6817/10502","state":["exists","up"]},{"osd":2,"uuid":"f09f00b4-e6cf-49ad-a53c-24bddd850d00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":10503},{"type":"v1","addr":"172.21.15.50:6816","nonce":10503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":10503},{"type":"v1","addr":"172.21.15.50:6820","nonce":10503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":10503},{"type":"v1","addr":"172.21.15.50:6825","nonce":10503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":10503},{"type":"v1","addr":"172.21.15.50:6823","nonce":10503}]},"public_addr":"172.21.15.50:6816/10503","cluster_addr":"172.21.15.50:6820/10503","heartbeat_back_addr":"172.21.15.50:6825/10503","heartbeat_front_addr":"172.21.15.50:6823/10503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:06.598 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 tell osd.0 flush_pg_stats 2020-05-24T13:49:07.226 INFO:teuthology.orchestra.run.smithi050.stdout:38654705669 2020-05-24T13:49:07.247 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 tell osd.1 flush_pg_stats 2020-05-24T13:49:07.952 INFO:teuthology.orchestra.run.smithi050.stdout:38654705668 2020-05-24T13:49:07.971 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 tell osd.2 flush_pg_stats 2020-05-24T13:49:08.551 INFO:teuthology.orchestra.run.smithi050.stdout:38654705668 2020-05-24T13:49:08.571 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd last-stat-seq osd.0 2020-05-24T13:49:09.432 INFO:teuthology.orchestra.run.smithi050.stdout:38654705669 2020-05-24T13:49:09.450 INFO:tasks.ceph.ceph_manager.c1:need seq 38654705669 got 38654705669 for osd.0 2020-05-24T13:49:09.450 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd last-stat-seq osd.1 2020-05-24T13:49:10.386 INFO:teuthology.orchestra.run.smithi050.stdout:38654705668 2020-05-24T13:49:10.402 INFO:tasks.ceph.ceph_manager.c1:need seq 38654705668 got 38654705668 for osd.1 2020-05-24T13:49:10.402 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 osd last-stat-seq osd.2 2020-05-24T13:49:11.295 INFO:teuthology.orchestra.run.smithi050.stdout:38654705668 2020-05-24T13:49:11.319 INFO:tasks.ceph.ceph_manager.c1:need seq 38654705668 got 38654705668 for osd.2 2020-05-24T13:49:11.320 INFO:tasks.ceph.ceph_manager.c1:waiting for clean 2020-05-24T13:49:11.320 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 pg dump --format=json 2020-05-24T13:49:12.155 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2020-05-24T13:49:12.155 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:12.173 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-05-24 13:49:10.685959","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"kb":314572800,"kb_used":3146088,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426712,"statfs":{"total":322122547200,"available":318900953088,"internally_reserved":3221225472,"allocated":344064,"data_stored":85863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.493385","last_change":"2020-05-24 13:49:01.032491","last_active":"2020-05-24 13:49:01.493385","last_peered":"2020-05-24 13:49:01.493385","last_clean":"2020-05-24 13:49:01.493385","last_became_active":"2020-05-24 13:49:00.983852","last_became_peered":"2020-05-24 13:49:00.983852","last_unstale":"2020-05-24 13:49:01.493385","last_undegraded":"2020-05-24 13:49:01.493385","last_fullsized":"2020-05-24 13:49:01.493385","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.497416","last_change":"2020-05-24 13:49:01.195023","last_active":"2020-05-24 13:49:01.497416","last_peered":"2020-05-24 13:49:01.497416","last_clean":"2020-05-24 13:49:01.497416","last_became_active":"2020-05-24 13:49:01.172753","last_became_peered":"2020-05-24 13:49:01.172753","last_unstale":"2020-05-24 13:49:01.497416","last_undegraded":"2020-05-24 13:49:01.497416","last_fullsized":"2020-05-24 13:49:01.497416","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2020-05-24 13:49:01.176771","last_change":"2020-05-24 13:49:01.098247","last_active":"2020-05-24 13:49:01.176771","last_peered":"2020-05-24 13:49:01.176771","last_clean":"2020-05-24 13:49:01.176771","last_became_active":"2020-05-24 13:49:01.049218","last_became_peered":"2020-05-24 13:49:01.049218","last_unstale":"2020-05-24 13:49:01.176771","last_undegraded":"2020-05-24 13:49:01.176771","last_fullsized":"2020-05-24 13:49:01.176771","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.487881","last_change":"2020-05-24 13:49:01.079342","last_active":"2020-05-24 13:49:01.487881","last_peered":"2020-05-24 13:49:01.487881","last_clean":"2020-05-24 13:49:01.487881","last_became_active":"2020-05-24 13:49:01.058842","last_became_peered":"2020-05-24 13:49:01.058842","last_unstale":"2020-05-24 13:49:01.487881","last_undegraded":"2020-05-24 13:49:01.487881","last_fullsized":"2020-05-24 13:49:01.487881","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.476551","last_change":"2020-05-24 13:49:01.163458","last_active":"2020-05-24 13:49:01.476551","last_peered":"2020-05-24 13:49:01.476551","last_clean":"2020-05-24 13:49:01.476551","last_became_active":"2020-05-24 13:49:01.138297","last_became_peered":"2020-05-24 13:49:01.138297","last_unstale":"2020-05-24 13:49:01.476551","last_undegraded":"2020-05-24 13:49:01.476551","last_fullsized":"2020-05-24 13:49:01.476551","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2020-05-24 13:49:01.151123","last_change":"2020-05-24 13:49:01.083719","last_active":"2020-05-24 13:49:01.151123","last_peered":"2020-05-24 13:49:01.151123","last_clean":"2020-05-24 13:49:01.151123","last_became_active":"2020-05-24 13:49:01.056499","last_became_peered":"2020-05-24 13:49:01.056499","last_unstale":"2020-05-24 13:49:01.151123","last_undegraded":"2020-05-24 13:49:01.151123","last_fullsized":"2020-05-24 13:49:01.151123","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.734812","last_change":"2020-05-24 13:49:01.697006","last_active":"2020-05-24 13:49:01.734812","last_peered":"2020-05-24 13:49:01.734812","last_clean":"2020-05-24 13:49:01.734812","last_became_active":"2020-05-24 13:49:01.664500","last_became_peered":"2020-05-24 13:49:01.664500","last_unstale":"2020-05-24 13:49:01.734812","last_undegraded":"2020-05-24 13:49:01.734812","last_fullsized":"2020-05-24 13:49:01.734812","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.482492","last_change":"2020-05-24 13:49:01.100484","last_active":"2020-05-24 13:49:01.482492","last_peered":"2020-05-24 13:49:01.482492","last_clean":"2020-05-24 13:49:01.482492","last_became_active":"2020-05-24 13:49:00.961223","last_became_peered":"2020-05-24 13:49:00.961223","last_unstale":"2020-05-24 13:49:01.482492","last_undegraded":"2020-05-24 13:49:01.482492","last_fullsized":"2020-05-24 13:49:01.482492","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}]}} 2020-05-24T13:49:12.175 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 pg dump --format=json 2020-05-24T13:49:13.088 INFO:teuthology.orchestra.run.smithi050.stderr:dumped all 2020-05-24T13:49:13.088 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:13.110 INFO:teuthology.orchestra.run.smithi050.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2020-05-24 13:49:12.686475","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"kb":314572800,"kb_used":3146088,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426712,"statfs":{"total":322122547200,"available":318900953088,"internally_reserved":3221225472,"allocated":344064,"data_stored":85863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.493385","last_change":"2020-05-24 13:49:01.032491","last_active":"2020-05-24 13:49:01.493385","last_peered":"2020-05-24 13:49:01.493385","last_clean":"2020-05-24 13:49:01.493385","last_became_active":"2020-05-24 13:49:00.983852","last_became_peered":"2020-05-24 13:49:00.983852","last_unstale":"2020-05-24 13:49:01.493385","last_undegraded":"2020-05-24 13:49:01.493385","last_fullsized":"2020-05-24 13:49:01.493385","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.497416","last_change":"2020-05-24 13:49:01.195023","last_active":"2020-05-24 13:49:01.497416","last_peered":"2020-05-24 13:49:01.497416","last_clean":"2020-05-24 13:49:01.497416","last_became_active":"2020-05-24 13:49:01.172753","last_became_peered":"2020-05-24 13:49:01.172753","last_unstale":"2020-05-24 13:49:01.497416","last_undegraded":"2020-05-24 13:49:01.497416","last_fullsized":"2020-05-24 13:49:01.497416","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2020-05-24 13:49:01.176771","last_change":"2020-05-24 13:49:01.098247","last_active":"2020-05-24 13:49:01.176771","last_peered":"2020-05-24 13:49:01.176771","last_clean":"2020-05-24 13:49:01.176771","last_became_active":"2020-05-24 13:49:01.049218","last_became_peered":"2020-05-24 13:49:01.049218","last_unstale":"2020-05-24 13:49:01.176771","last_undegraded":"2020-05-24 13:49:01.176771","last_fullsized":"2020-05-24 13:49:01.176771","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.487881","last_change":"2020-05-24 13:49:01.079342","last_active":"2020-05-24 13:49:01.487881","last_peered":"2020-05-24 13:49:01.487881","last_clean":"2020-05-24 13:49:01.487881","last_became_active":"2020-05-24 13:49:01.058842","last_became_peered":"2020-05-24 13:49:01.058842","last_unstale":"2020-05-24 13:49:01.487881","last_undegraded":"2020-05-24 13:49:01.487881","last_fullsized":"2020-05-24 13:49:01.487881","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.476551","last_change":"2020-05-24 13:49:01.163458","last_active":"2020-05-24 13:49:01.476551","last_peered":"2020-05-24 13:49:01.476551","last_clean":"2020-05-24 13:49:01.476551","last_became_active":"2020-05-24 13:49:01.138297","last_became_peered":"2020-05-24 13:49:01.138297","last_unstale":"2020-05-24 13:49:01.476551","last_undegraded":"2020-05-24 13:49:01.476551","last_fullsized":"2020-05-24 13:49:01.476551","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"9","reported_epoch":"11","state":"active+clean","last_fresh":"2020-05-24 13:49:01.151123","last_change":"2020-05-24 13:49:01.083719","last_active":"2020-05-24 13:49:01.151123","last_peered":"2020-05-24 13:49:01.151123","last_clean":"2020-05-24 13:49:01.151123","last_became_active":"2020-05-24 13:49:01.056499","last_became_peered":"2020-05-24 13:49:01.056499","last_unstale":"2020-05-24 13:49:01.151123","last_undegraded":"2020-05-24 13:49:01.151123","last_fullsized":"2020-05-24 13:49:01.151123","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.734812","last_change":"2020-05-24 13:49:01.697006","last_active":"2020-05-24 13:49:01.734812","last_peered":"2020-05-24 13:49:01.734812","last_clean":"2020-05-24 13:49:01.734812","last_became_active":"2020-05-24 13:49:01.664500","last_became_peered":"2020-05-24 13:49:01.664500","last_unstale":"2020-05-24 13:49:01.734812","last_undegraded":"2020-05-24 13:49:01.734812","last_fullsized":"2020-05-24 13:49:01.734812","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:49:01.482492","last_change":"2020-05-24 13:49:01.100484","last_active":"2020-05-24 13:49:01.482492","last_peered":"2020-05-24 13:49:01.482492","last_clean":"2020-05-24 13:49:01.482492","last_became_active":"2020-05-24 13:49:00.961223","last_became_peered":"2020-05-24 13:49:00.961223","last_unstale":"2020-05-24 13:49:01.482492","last_undegraded":"2020-05-24 13:49:01.482492","last_fullsized":"2020-05-24 13:49:01.482492","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:48:59.148549","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:48:59.148549","last_clean_scrub_stamp":"2020-05-24 13:48:59.148549","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":28621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}]}} 2020-05-24T13:49:13.111 INFO:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c1 status --format=json-pretty 2020-05-24T13:49:14.230 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:14.230 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2020-05-24T13:49:14.230 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "c458a90b-5841-48fd-9e05-3dd82e2be174", 2020-05-24T13:49:14.231 INFO:teuthology.orchestra.run.smithi050.stdout: "health": { 2020-05-24T13:49:14.231 INFO:teuthology.orchestra.run.smithi050.stdout: "checks": {}, 2020-05-24T13:49:14.231 INFO:teuthology.orchestra.run.smithi050.stdout: "status": "HEALTH_OK" 2020-05-24T13:49:14.231 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.231 INFO:teuthology.orchestra.run.smithi050.stdout: "election_epoch": 3, 2020-05-24T13:49:14.231 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum": [ 2020-05-24T13:49:14.232 INFO:teuthology.orchestra.run.smithi050.stdout: 0 2020-05-24T13:49:14.232 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2020-05-24T13:49:14.232 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_names": [ 2020-05-24T13:49:14.232 INFO:teuthology.orchestra.run.smithi050.stdout: "a" 2020-05-24T13:49:14.232 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2020-05-24T13:49:14.233 INFO:teuthology.orchestra.run.smithi050.stdout: "quorum_age": 40, 2020-05-24T13:49:14.233 INFO:teuthology.orchestra.run.smithi050.stdout: "monmap": { 2020-05-24T13:49:14.233 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2020-05-24T13:49:14.233 INFO:teuthology.orchestra.run.smithi050.stdout: "fsid": "c458a90b-5841-48fd-9e05-3dd82e2be174", 2020-05-24T13:49:14.233 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "2020-05-24 13:48:11.108452", 2020-05-24T13:49:14.234 INFO:teuthology.orchestra.run.smithi050.stdout: "created": "2020-05-24 13:48:11.108452", 2020-05-24T13:49:14.234 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release": 14, 2020-05-24T13:49:14.235 INFO:teuthology.orchestra.run.smithi050.stdout: "min_mon_release_name": "nautilus", 2020-05-24T13:49:14.235 INFO:teuthology.orchestra.run.smithi050.stdout: "features": { 2020-05-24T13:49:14.235 INFO:teuthology.orchestra.run.smithi050.stdout: "persistent": [ 2020-05-24T13:49:14.236 INFO:teuthology.orchestra.run.smithi050.stdout: "kraken", 2020-05-24T13:49:14.236 INFO:teuthology.orchestra.run.smithi050.stdout: "luminous", 2020-05-24T13:49:14.236 INFO:teuthology.orchestra.run.smithi050.stdout: "mimic", 2020-05-24T13:49:14.236 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap-prune", 2020-05-24T13:49:14.236 INFO:teuthology.orchestra.run.smithi050.stdout: "nautilus" 2020-05-24T13:49:14.237 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2020-05-24T13:49:14.237 INFO:teuthology.orchestra.run.smithi050.stdout: "optional": [] 2020-05-24T13:49:14.237 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.237 INFO:teuthology.orchestra.run.smithi050.stdout: "mons": [ 2020-05-24T13:49:14.237 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.238 INFO:teuthology.orchestra.run.smithi050.stdout: "rank": 0, 2020-05-24T13:49:14.238 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "a", 2020-05-24T13:49:14.238 INFO:teuthology.orchestra.run.smithi050.stdout: "public_addrs": { 2020-05-24T13:49:14.238 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2020-05-24T13:49:14.238 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.238 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2020-05-24T13:49:14.239 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:3300", 2020-05-24T13:49:14.239 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 0 2020-05-24T13:49:14.239 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.239 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.239 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2020-05-24T13:49:14.240 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6789", 2020-05-24T13:49:14.240 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 0 2020-05-24T13:49:14.240 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.240 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2020-05-24T13:49:14.240 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.240 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6789/0", 2020-05-24T13:49:14.241 INFO:teuthology.orchestra.run.smithi050.stdout: "public_addr": "172.21.15.50:6789/0" 2020-05-24T13:49:14.241 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.241 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2020-05-24T13:49:14.241 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.241 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2020-05-24T13:49:14.242 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap": { 2020-05-24T13:49:14.242 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 13, 2020-05-24T13:49:14.242 INFO:teuthology.orchestra.run.smithi050.stdout: "num_osds": 3, 2020-05-24T13:49:14.242 INFO:teuthology.orchestra.run.smithi050.stdout: "num_up_osds": 3, 2020-05-24T13:49:14.242 INFO:teuthology.orchestra.run.smithi050.stdout: "num_in_osds": 3, 2020-05-24T13:49:14.243 INFO:teuthology.orchestra.run.smithi050.stdout: "num_remapped_pgs": 0 2020-05-24T13:49:14.243 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.243 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.243 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap": { 2020-05-24T13:49:14.243 INFO:teuthology.orchestra.run.smithi050.stdout: "pgs_by_state": [ 2020-05-24T13:49:14.243 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.244 INFO:teuthology.orchestra.run.smithi050.stdout: "state_name": "active+clean", 2020-05-24T13:49:14.244 INFO:teuthology.orchestra.run.smithi050.stdout: "count": 8 2020-05-24T13:49:14.244 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2020-05-24T13:49:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pgs": 8, 2020-05-24T13:49:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: "num_pools": 1, 2020-05-24T13:49:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: "num_objects": 0, 2020-05-24T13:49:14.245 INFO:teuthology.orchestra.run.smithi050.stdout: "data_bytes": 0, 2020-05-24T13:49:14.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_used": 3221594112, 2020-05-24T13:49:14.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_avail": 318900953088, 2020-05-24T13:49:14.246 INFO:teuthology.orchestra.run.smithi050.stdout: "bytes_total": 322122547200 2020-05-24T13:49:14.246 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.247 INFO:teuthology.orchestra.run.smithi050.stdout: "fsmap": { 2020-05-24T13:49:14.247 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2020-05-24T13:49:14.247 INFO:teuthology.orchestra.run.smithi050.stdout: "by_rank": [], 2020-05-24T13:49:14.247 INFO:teuthology.orchestra.run.smithi050.stdout: "up:standby": 0 2020-05-24T13:49:14.247 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.248 INFO:teuthology.orchestra.run.smithi050.stdout: "mgrmap": { 2020-05-24T13:49:14.248 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 3, 2020-05-24T13:49:14.248 INFO:teuthology.orchestra.run.smithi050.stdout: "active_gid": 4100, 2020-05-24T13:49:14.248 INFO:teuthology.orchestra.run.smithi050.stdout: "active_name": "x", 2020-05-24T13:49:14.248 INFO:teuthology.orchestra.run.smithi050.stdout: "active_addrs": { 2020-05-24T13:49:14.248 INFO:teuthology.orchestra.run.smithi050.stdout: "addrvec": [ 2020-05-24T13:49:14.249 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.249 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v2", 2020-05-24T13:49:14.249 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6800", 2020-05-24T13:49:14.249 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 9595 2020-05-24T13:49:14.249 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.250 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.250 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "v1", 2020-05-24T13:49:14.250 INFO:teuthology.orchestra.run.smithi050.stdout: "addr": "172.21.15.50:6801", 2020-05-24T13:49:14.250 INFO:teuthology.orchestra.run.smithi050.stdout: "nonce": 9595 2020-05-24T13:49:14.251 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.251 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2020-05-24T13:49:14.251 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.251 INFO:teuthology.orchestra.run.smithi050.stdout: "active_addr": "172.21.15.50:6801/9595", 2020-05-24T13:49:14.251 INFO:teuthology.orchestra.run.smithi050.stdout: "active_change": "2020-05-24 13:48:38.492669", 2020-05-24T13:49:14.251 INFO:teuthology.orchestra.run.smithi050.stdout: "available": true, 2020-05-24T13:49:14.252 INFO:teuthology.orchestra.run.smithi050.stdout: "standbys": [], 2020-05-24T13:49:14.252 INFO:teuthology.orchestra.run.smithi050.stdout: "modules": [ 2020-05-24T13:49:14.252 INFO:teuthology.orchestra.run.smithi050.stdout: "iostat", 2020-05-24T13:49:14.252 INFO:teuthology.orchestra.run.smithi050.stdout: "restful" 2020-05-24T13:49:14.252 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2020-05-24T13:49:14.253 INFO:teuthology.orchestra.run.smithi050.stdout: "available_modules": [ 2020-05-24T13:49:14.253 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.253 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "alerts", 2020-05-24T13:49:14.253 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.253 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.253 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.254 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2020-05-24T13:49:14.254 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2020-05-24T13:49:14.254 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.254 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.255 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.255 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2020-05-24T13:49:14.255 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.255 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.255 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.256 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "How frequently to reexamine health status", 2020-05-24T13:49:14.256 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.257 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.257 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.257 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.257 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_destination": { 2020-05-24T13:49:14.258 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_destination", 2020-05-24T13:49:14.258 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.258 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.258 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.258 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.259 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.259 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.259 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.259 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email address to send alerts to", 2020-05-24T13:49:14.259 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.260 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.260 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.260 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.260 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_from_name": { 2020-05-24T13:49:14.260 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_from_name", 2020-05-24T13:49:14.260 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.261 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.261 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.261 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "Ceph", 2020-05-24T13:49:14.261 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.261 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.262 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.262 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Email From: name", 2020-05-24T13:49:14.262 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.262 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.262 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.262 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.263 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_host": { 2020-05-24T13:49:14.263 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_host", 2020-05-24T13:49:14.263 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.263 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.263 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.264 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.264 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.264 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.264 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.264 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP server", 2020-05-24T13:49:14.265 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.265 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.265 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.265 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.265 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_password": { 2020-05-24T13:49:14.266 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_password", 2020-05-24T13:49:14.266 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.266 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.267 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.267 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.267 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.267 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.267 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.267 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Password to authenticate with", 2020-05-24T13:49:14.268 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.268 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.268 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.268 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.268 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_port": { 2020-05-24T13:49:14.269 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_port", 2020-05-24T13:49:14.269 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.269 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.269 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.269 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "465", 2020-05-24T13:49:14.269 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.270 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.270 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.270 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP port", 2020-05-24T13:49:14.270 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.270 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.271 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.271 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.271 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_sender": { 2020-05-24T13:49:14.271 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_sender", 2020-05-24T13:49:14.272 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.272 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.272 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.272 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.272 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.272 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.273 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.273 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "SMTP envelope sender", 2020-05-24T13:49:14.273 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.273 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.273 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.273 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.274 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_ssl": { 2020-05-24T13:49:14.274 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_ssl", 2020-05-24T13:49:14.274 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.274 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.274 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.275 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.275 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.275 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.275 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.276 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Use SSL to connect to SMTP server", 2020-05-24T13:49:14.276 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.276 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.276 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.276 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.277 INFO:teuthology.orchestra.run.smithi050.stdout: "smtp_user": { 2020-05-24T13:49:14.277 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "smtp_user", 2020-05-24T13:49:14.277 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.277 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.278 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.278 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.278 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.278 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.278 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.279 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "User to authenticate as", 2020-05-24T13:49:14.279 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.279 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.279 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.279 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.279 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.280 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.280 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.280 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ansible", 2020-05-24T13:49:14.280 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.280 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.281 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.281 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2020-05-24T13:49:14.281 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2020-05-24T13:49:14.281 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.282 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.282 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.282 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.282 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.282 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.282 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.283 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.283 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.283 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.283 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.283 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.284 INFO:teuthology.orchestra.run.smithi050.stdout: "server_url": { 2020-05-24T13:49:14.284 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_url", 2020-05-24T13:49:14.284 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.284 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.284 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.284 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.285 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.285 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.285 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.285 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.286 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.286 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.286 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.286 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.286 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2020-05-24T13:49:14.286 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2020-05-24T13:49:14.287 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.287 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.287 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.288 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.288 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.288 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.288 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.288 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.289 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.289 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.289 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.289 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.289 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_server": { 2020-05-24T13:49:14.289 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_server", 2020-05-24T13:49:14.290 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.290 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.290 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.290 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.290 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.290 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.291 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.291 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.291 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.291 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.291 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.292 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.292 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.292 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.292 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.293 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "balancer", 2020-05-24T13:49:14.293 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.293 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.293 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.293 INFO:teuthology.orchestra.run.smithi050.stdout: "active": { 2020-05-24T13:49:14.294 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "active", 2020-05-24T13:49:14.294 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.294 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.294 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.294 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2020-05-24T13:49:14.295 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.295 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.295 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.295 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically balance PGs across cluster", 2020-05-24T13:49:14.296 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.296 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.296 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.296 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.296 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_time": { 2020-05-24T13:49:14.297 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_time", 2020-05-24T13:49:14.297 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.297 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.297 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.298 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0000", 2020-05-24T13:49:14.298 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.298 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.298 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.298 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "beginning time of day to automatically balance", 2020-05-24T13:49:14.299 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2020-05-24T13:49:14.299 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.299 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.299 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.300 INFO:teuthology.orchestra.run.smithi050.stdout: "begin_weekday": { 2020-05-24T13:49:14.300 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "begin_weekday", 2020-05-24T13:49:14.300 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2020-05-24T13:49:14.300 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.300 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.301 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2020-05-24T13:49:14.301 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2020-05-24T13:49:14.301 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "7", 2020-05-24T13:49:14.301 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.301 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2020-05-24T13:49:14.301 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-05-24T13:49:14.302 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.302 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.302 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.302 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_max_iterations": { 2020-05-24T13:49:14.302 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_max_iterations", 2020-05-24T13:49:14.303 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2020-05-24T13:49:14.303 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.303 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.303 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "25", 2020-05-24T13:49:14.303 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2020-05-24T13:49:14.303 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "250", 2020-05-24T13:49:14.304 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.304 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum number of iterations to attempt optimization", 2020-05-24T13:49:14.304 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.304 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.304 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.304 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.305 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_metrics": { 2020-05-24T13:49:14.305 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_metrics", 2020-05-24T13:49:14.305 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.305 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.306 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.306 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "pgs,objects,bytes", 2020-05-24T13:49:14.306 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.306 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.306 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.307 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "metrics with which to calculate OSD utilization", 2020-05-24T13:49:14.307 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2020-05-24T13:49:14.307 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.307 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.307 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.308 INFO:teuthology.orchestra.run.smithi050.stdout: "crush_compat_step": { 2020-05-24T13:49:14.308 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crush_compat_step", 2020-05-24T13:49:14.309 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2020-05-24T13:49:14.309 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.309 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.309 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0.5", 2020-05-24T13:49:14.309 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0.001", 2020-05-24T13:49:14.309 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "0.999", 2020-05-24T13:49:14.310 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.310 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "aggressiveness of optimization", 2020-05-24T13:49:14.310 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2020-05-24T13:49:14.310 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.310 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.311 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.311 INFO:teuthology.orchestra.run.smithi050.stdout: "end_time": { 2020-05-24T13:49:14.311 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_time", 2020-05-24T13:49:14.311 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.311 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.312 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.312 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2400", 2020-05-24T13:49:14.312 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.312 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.312 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.313 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "ending time of day to automatically balance", 2020-05-24T13:49:14.313 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "This is a time of day in the format HHMM.", 2020-05-24T13:49:14.313 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.313 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.313 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.314 INFO:teuthology.orchestra.run.smithi050.stdout: "end_weekday": { 2020-05-24T13:49:14.314 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "end_weekday", 2020-05-24T13:49:14.314 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2020-05-24T13:49:14.314 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.314 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.315 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7", 2020-05-24T13:49:14.315 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "0", 2020-05-24T13:49:14.315 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "7", 2020-05-24T13:49:14.315 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.315 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2020-05-24T13:49:14.315 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-05-24T13:49:14.316 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.316 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.316 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.316 INFO:teuthology.orchestra.run.smithi050.stdout: "min_score": { 2020-05-24T13:49:14.317 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_score", 2020-05-24T13:49:14.317 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2020-05-24T13:49:14.317 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.317 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.317 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "0", 2020-05-24T13:49:14.318 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.318 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.318 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.319 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "minimum score, below which no optimization is attempted", 2020-05-24T13:49:14.319 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.319 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.319 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.319 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.319 INFO:teuthology.orchestra.run.smithi050.stdout: "mode": { 2020-05-24T13:49:14.320 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mode", 2020-05-24T13:49:14.320 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.320 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.320 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.320 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "none", 2020-05-24T13:49:14.321 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.321 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.321 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2020-05-24T13:49:14.321 INFO:teuthology.orchestra.run.smithi050.stdout: "crush-compat", 2020-05-24T13:49:14.321 INFO:teuthology.orchestra.run.smithi050.stdout: "none", 2020-05-24T13:49:14.321 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap" 2020-05-24T13:49:14.322 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2020-05-24T13:49:14.322 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Balancer mode", 2020-05-24T13:49:14.322 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.322 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.322 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.323 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.323 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_ids": { 2020-05-24T13:49:14.323 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_ids", 2020-05-24T13:49:14.323 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.323 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.324 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.324 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.324 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.324 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.324 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.324 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "pools which the automatic balancing will be limited to", 2020-05-24T13:49:14.325 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.325 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.325 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.325 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.325 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2020-05-24T13:49:14.326 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2020-05-24T13:49:14.326 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.326 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.326 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.327 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2020-05-24T13:49:14.327 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.327 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.327 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.327 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and attempt optimization", 2020-05-24T13:49:14.327 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.328 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.328 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.328 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.329 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_deviation": { 2020-05-24T13:49:14.329 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_deviation", 2020-05-24T13:49:14.329 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.329 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.329 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2020-05-24T13:49:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "1", 2020-05-24T13:49:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "deviation below which no optimization is attempted", 2020-05-24T13:49:14.331 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2020-05-24T13:49:14.331 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.331 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.331 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.331 INFO:teuthology.orchestra.run.smithi050.stdout: "upmap_max_iterations": { 2020-05-24T13:49:14.331 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "upmap_max_iterations", 2020-05-24T13:49:14.332 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "uint", 2020-05-24T13:49:14.332 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.332 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.332 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10", 2020-05-24T13:49:14.332 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.332 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.333 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.333 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "maximum upmap optimization iterations", 2020-05-24T13:49:14.333 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.333 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.333 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.334 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.334 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.334 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.335 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.335 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crash", 2020-05-24T13:49:14.335 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.335 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.335 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.336 INFO:teuthology.orchestra.run.smithi050.stdout: "retain_interval": { 2020-05-24T13:49:14.336 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retain_interval", 2020-05-24T13:49:14.336 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.336 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.337 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.337 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31536000", 2020-05-24T13:49:14.337 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.337 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.337 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.338 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain crashes before pruning them", 2020-05-24T13:49:14.338 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.338 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.338 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.338 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.338 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_recent_interval": { 2020-05-24T13:49:14.339 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_recent_interval", 2020-05-24T13:49:14.339 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.339 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.340 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.340 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1209600", 2020-05-24T13:49:14.340 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.340 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.340 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.341 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "time interval in which to warn about recent crashes", 2020-05-24T13:49:14.341 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.341 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.341 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.341 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.341 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.342 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.342 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.342 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "dashboard", 2020-05-24T13:49:14.342 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.342 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.342 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.343 INFO:teuthology.orchestra.run.smithi050.stdout: "ALERTMANAGER_API_HOST": { 2020-05-24T13:49:14.343 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ALERTMANAGER_API_HOST", 2020-05-24T13:49:14.343 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.343 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.343 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.344 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.344 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.344 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.344 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.344 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.344 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.345 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.345 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.345 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.345 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_ENABLED": { 2020-05-24T13:49:14.345 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_ENABLED", 2020-05-24T13:49:14.346 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.346 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.346 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.346 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2020-05-24T13:49:14.346 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.347 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.347 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.347 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.347 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.347 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.348 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.348 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.348 INFO:teuthology.orchestra.run.smithi050.stdout: "AUDIT_API_LOG_PAYLOAD": { 2020-05-24T13:49:14.348 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2020-05-24T13:49:14.348 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.348 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.349 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.349 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.349 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.350 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.350 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.350 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.350 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.350 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.351 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.351 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.351 INFO:teuthology.orchestra.run.smithi050.stdout: "ENABLE_BROWSABLE_API": { 2020-05-24T13:49:14.351 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ENABLE_BROWSABLE_API", 2020-05-24T13:49:14.352 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.352 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.352 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.352 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.352 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.352 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.353 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.353 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.353 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.353 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.353 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.354 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.354 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_cephfs": { 2020-05-24T13:49:14.354 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_cephfs", 2020-05-24T13:49:14.354 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.354 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.354 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.355 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.355 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.355 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.355 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.355 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.356 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.356 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.356 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.356 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.356 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_iscsi": { 2020-05-24T13:49:14.357 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_iscsi", 2020-05-24T13:49:14.357 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.358 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.358 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.358 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.358 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.358 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.359 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.359 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.359 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.359 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.360 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.360 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.360 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_mirroring": { 2020-05-24T13:49:14.360 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_mirroring", 2020-05-24T13:49:14.360 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.361 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.361 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.361 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.361 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.361 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.362 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.362 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.362 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.362 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.362 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.363 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.363 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_rbd": { 2020-05-24T13:49:14.363 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_rbd", 2020-05-24T13:49:14.363 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.363 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.363 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.364 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.364 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.364 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.364 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.364 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.365 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.365 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.365 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.365 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.365 INFO:teuthology.orchestra.run.smithi050.stdout: "FEATURE_TOGGLE_rgw": { 2020-05-24T13:49:14.366 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "FEATURE_TOGGLE_rgw", 2020-05-24T13:49:14.366 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.366 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.366 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.366 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.366 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.367 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.367 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.367 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.367 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.368 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.368 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.368 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.368 INFO:teuthology.orchestra.run.smithi050.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2020-05-24T13:49:14.368 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2020-05-24T13:49:14.369 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.369 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.369 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.369 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.369 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.369 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.370 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.370 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.370 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.371 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.371 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.371 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.371 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_PASSWORD": { 2020-05-24T13:49:14.371 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_PASSWORD", 2020-05-24T13:49:14.372 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.372 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.372 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.372 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2020-05-24T13:49:14.373 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.373 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.373 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.373 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.373 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.373 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.374 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.374 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.374 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_SSL_VERIFY": { 2020-05-24T13:49:14.374 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2020-05-24T13:49:14.374 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.375 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.375 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.375 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.375 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.375 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.375 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.376 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.376 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.376 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.376 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.376 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.377 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_URL": { 2020-05-24T13:49:14.377 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_URL", 2020-05-24T13:49:14.377 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.377 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.378 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.378 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.378 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.378 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.378 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.378 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.379 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.379 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.379 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.379 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.379 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_API_USERNAME": { 2020-05-24T13:49:14.380 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_API_USERNAME", 2020-05-24T13:49:14.380 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.380 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.380 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.381 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2020-05-24T13:49:14.381 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.381 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.381 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.382 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.382 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.382 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.382 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.382 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.383 INFO:teuthology.orchestra.run.smithi050.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2020-05-24T13:49:14.383 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2020-05-24T13:49:14.383 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.383 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.383 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.384 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2020-05-24T13:49:14.384 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.384 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.384 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.384 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.385 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.385 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.385 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.385 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.385 INFO:teuthology.orchestra.run.smithi050.stdout: "ISCSI_API_SSL_VERIFICATION": { 2020-05-24T13:49:14.386 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2020-05-24T13:49:14.386 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.386 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.386 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.386 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.386 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.387 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.387 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.387 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.387 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.387 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.388 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.388 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.388 INFO:teuthology.orchestra.run.smithi050.stdout: "PROMETHEUS_API_HOST": { 2020-05-24T13:49:14.388 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "PROMETHEUS_API_HOST", 2020-05-24T13:49:14.389 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.389 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.389 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.389 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.389 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.389 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.390 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.390 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.390 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.390 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.390 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.391 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.391 INFO:teuthology.orchestra.run.smithi050.stdout: "REST_REQUESTS_TIMEOUT": { 2020-05-24T13:49:14.391 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "REST_REQUESTS_TIMEOUT", 2020-05-24T13:49:14.392 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.392 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.392 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.392 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "45", 2020-05-24T13:49:14.392 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.393 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.393 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.393 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.393 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.394 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.394 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.394 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.394 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ACCESS_KEY": { 2020-05-24T13:49:14.394 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ACCESS_KEY", 2020-05-24T13:49:14.395 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.395 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.395 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.395 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.395 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.396 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.396 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.396 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.396 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.396 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.396 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.397 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.397 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_ADMIN_RESOURCE": { 2020-05-24T13:49:14.397 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2020-05-24T13:49:14.397 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.397 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.397 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.398 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "admin", 2020-05-24T13:49:14.398 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.398 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.399 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.399 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.399 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.399 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.399 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.399 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.400 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_HOST": { 2020-05-24T13:49:14.400 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_HOST", 2020-05-24T13:49:14.400 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.400 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.400 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.401 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.401 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.401 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.402 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.402 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.402 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.402 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.402 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.403 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.403 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_PORT": { 2020-05-24T13:49:14.403 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_PORT", 2020-05-24T13:49:14.403 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.403 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.403 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.404 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "80", 2020-05-24T13:49:14.404 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.404 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.404 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.404 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.405 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.405 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.405 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.405 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.405 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SCHEME": { 2020-05-24T13:49:14.405 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SCHEME", 2020-05-24T13:49:14.406 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.406 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.406 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.406 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "http", 2020-05-24T13:49:14.406 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.407 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.407 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.407 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.407 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.407 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.407 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.408 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.408 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SECRET_KEY": { 2020-05-24T13:49:14.408 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SECRET_KEY", 2020-05-24T13:49:14.409 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.409 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.409 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.409 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.411 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.411 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.411 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_SSL_VERIFY": { 2020-05-24T13:49:14.411 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_SSL_VERIFY", 2020-05-24T13:49:14.412 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.412 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.412 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.412 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.413 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.413 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.413 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.413 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.413 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.414 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.414 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.414 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.414 INFO:teuthology.orchestra.run.smithi050.stdout: "RGW_API_USER_ID": { 2020-05-24T13:49:14.414 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "RGW_API_USER_ID", 2020-05-24T13:49:14.415 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.415 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.415 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.415 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.415 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.416 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.416 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.416 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.416 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.416 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.417 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.417 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.417 INFO:teuthology.orchestra.run.smithi050.stdout: "crt_file": { 2020-05-24T13:49:14.417 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "crt_file", 2020-05-24T13:49:14.417 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.417 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.418 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.418 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.418 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.418 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.419 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.419 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.419 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.419 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.419 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.420 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.420 INFO:teuthology.orchestra.run.smithi050.stdout: "debug": { 2020-05-24T13:49:14.420 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "debug", 2020-05-24T13:49:14.420 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.420 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.421 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.421 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2020-05-24T13:49:14.421 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.421 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.421 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.421 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Enable/disable debug options", 2020-05-24T13:49:14.422 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.422 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.422 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.423 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.423 INFO:teuthology.orchestra.run.smithi050.stdout: "jwt_token_ttl": { 2020-05-24T13:49:14.423 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "jwt_token_ttl", 2020-05-24T13:49:14.423 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.423 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.424 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.424 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "28800", 2020-05-24T13:49:14.424 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.424 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.424 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.424 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.425 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.425 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.425 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.425 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.425 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2020-05-24T13:49:14.426 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2020-05-24T13:49:14.426 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.426 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.426 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.426 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.427 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.427 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.427 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.427 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.427 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.427 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.428 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.428 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.428 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2020-05-24T13:49:14.428 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2020-05-24T13:49:14.428 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.429 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.429 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.429 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.429 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.430 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.430 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.430 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.430 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.430 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.431 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.431 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.431 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2020-05-24T13:49:14.431 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2020-05-24T13:49:14.431 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.431 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.432 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.432 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "::", 2020-05-24T13:49:14.432 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.433 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.433 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.433 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.433 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.433 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.434 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.434 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.434 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2020-05-24T13:49:14.434 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2020-05-24T13:49:14.434 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.434 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.435 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.435 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8080", 2020-05-24T13:49:14.435 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.436 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.436 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.436 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.436 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.436 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.437 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.437 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.437 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2020-05-24T13:49:14.437 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2020-05-24T13:49:14.437 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.437 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.438 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.438 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.438 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.438 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.438 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.439 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.439 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.439 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.439 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.440 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.440 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl_server_port": { 2020-05-24T13:49:14.440 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl_server_port", 2020-05-24T13:49:14.440 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.440 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.441 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.441 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8443", 2020-05-24T13:49:14.441 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.441 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.441 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.441 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.442 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.442 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.442 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.442 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.442 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_behaviour": { 2020-05-24T13:49:14.443 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_behaviour", 2020-05-24T13:49:14.443 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.443 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.444 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.444 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "redirect", 2020-05-24T13:49:14.444 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.444 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.444 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [ 2020-05-24T13:49:14.444 INFO:teuthology.orchestra.run.smithi050.stdout: "error", 2020-05-24T13:49:14.445 INFO:teuthology.orchestra.run.smithi050.stdout: "redirect" 2020-05-24T13:49:14.445 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2020-05-24T13:49:14.445 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.445 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.445 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.446 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.446 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.446 INFO:teuthology.orchestra.run.smithi050.stdout: "standby_error_status_code": { 2020-05-24T13:49:14.446 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "standby_error_status_code", 2020-05-24T13:49:14.446 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.446 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.447 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.447 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "500", 2020-05-24T13:49:14.447 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "400", 2020-05-24T13:49:14.447 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "599", 2020-05-24T13:49:14.447 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.448 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.448 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.448 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.448 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.448 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.448 INFO:teuthology.orchestra.run.smithi050.stdout: "url_prefix": { 2020-05-24T13:49:14.449 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url_prefix", 2020-05-24T13:49:14.449 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.449 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.449 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.450 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.450 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.450 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.450 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.450 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.451 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.451 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.451 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.451 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.452 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2020-05-24T13:49:14.452 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2020-05-24T13:49:14.452 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.452 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.452 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.453 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.453 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.453 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.454 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.454 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.454 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.454 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.454 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.455 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.455 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.455 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.455 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.455 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "deepsea", 2020-05-24T13:49:14.455 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.456 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.456 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.456 INFO:teuthology.orchestra.run.smithi050.stdout: "salt_api_eauth": { 2020-05-24T13:49:14.456 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "salt_api_eauth", 2020-05-24T13:49:14.456 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.457 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.457 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.457 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "sharedsecret", 2020-05-24T13:49:14.457 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.457 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.457 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.458 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.458 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.458 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.458 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.458 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.459 INFO:teuthology.orchestra.run.smithi050.stdout: "salt_api_password": { 2020-05-24T13:49:14.459 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "salt_api_password", 2020-05-24T13:49:14.459 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.459 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.459 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.459 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.460 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.460 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.460 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.460 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.461 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.461 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.461 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.461 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.461 INFO:teuthology.orchestra.run.smithi050.stdout: "salt_api_url": { 2020-05-24T13:49:14.462 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "salt_api_url", 2020-05-24T13:49:14.462 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.462 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.462 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.462 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.462 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.463 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.463 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.463 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.463 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.464 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.464 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.464 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.464 INFO:teuthology.orchestra.run.smithi050.stdout: "salt_api_username": { 2020-05-24T13:49:14.465 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "salt_api_username", 2020-05-24T13:49:14.465 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.465 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.465 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.465 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.465 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.466 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.466 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.466 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.466 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.466 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.467 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.467 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.467 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.467 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.467 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.467 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "devicehealth", 2020-05-24T13:49:14.468 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.468 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.468 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.468 INFO:teuthology.orchestra.run.smithi050.stdout: "enable_monitoring": { 2020-05-24T13:49:14.468 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enable_monitoring", 2020-05-24T13:49:14.469 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.469 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.469 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.469 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2020-05-24T13:49:14.469 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.469 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.470 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.470 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "monitor device health metrics", 2020-05-24T13:49:14.470 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.471 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.471 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.471 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.471 INFO:teuthology.orchestra.run.smithi050.stdout: "mark_out_threshold": { 2020-05-24T13:49:14.471 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "mark_out_threshold", 2020-05-24T13:49:14.472 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.472 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.472 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.472 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "2419200", 2020-05-24T13:49:14.472 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.473 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.473 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.473 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "automatically mark OSD if it may fail before this long", 2020-05-24T13:49:14.473 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.474 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.474 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.474 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.475 INFO:teuthology.orchestra.run.smithi050.stdout: "pool_name": { 2020-05-24T13:49:14.475 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pool_name", 2020-05-24T13:49:14.475 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.475 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.475 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.475 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "device_health_metrics", 2020-05-24T13:49:14.476 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.476 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.476 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.476 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name of pool in which to store device health metrics", 2020-05-24T13:49:14.476 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.477 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.477 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.477 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.477 INFO:teuthology.orchestra.run.smithi050.stdout: "retention_period": { 2020-05-24T13:49:14.477 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "retention_period", 2020-05-24T13:49:14.477 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.478 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.478 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.478 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15552000", 2020-05-24T13:49:14.478 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.478 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.479 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.479 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how long to retain device health metrics", 2020-05-24T13:49:14.479 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.479 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.479 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.479 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.480 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_frequency": { 2020-05-24T13:49:14.480 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_frequency", 2020-05-24T13:49:14.480 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.480 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.481 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.481 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2020-05-24T13:49:14.481 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.481 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.481 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.482 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to scrape device health metrics", 2020-05-24T13:49:14.482 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.482 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.482 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.482 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.483 INFO:teuthology.orchestra.run.smithi050.stdout: "self_heal": { 2020-05-24T13:49:14.483 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "self_heal", 2020-05-24T13:49:14.483 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.483 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.484 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.484 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.484 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.484 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.485 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.485 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "preemptively heal cluster around devices that may fail", 2020-05-24T13:49:14.485 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.485 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.486 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.486 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.486 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2020-05-24T13:49:14.486 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2020-05-24T13:49:14.486 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.486 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.487 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.487 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2020-05-24T13:49:14.487 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.487 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.487 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.488 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to wake up and check device health", 2020-05-24T13:49:14.488 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.488 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.488 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.488 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.488 INFO:teuthology.orchestra.run.smithi050.stdout: "warn_threshold": { 2020-05-24T13:49:14.489 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "warn_threshold", 2020-05-24T13:49:14.489 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.489 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.489 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.489 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "7257600", 2020-05-24T13:49:14.490 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.490 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.490 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.490 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "raise health warning if OSD may fail before this long", 2020-05-24T13:49:14.490 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.491 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.491 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.491 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.491 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.492 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.492 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.492 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_cloud", 2020-05-24T13:49:14.492 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.492 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.493 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.493 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_cert_context": { 2020-05-24T13:49:14.493 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_cert_context", 2020-05-24T13:49:14.493 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.493 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.494 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.494 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.494 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.494 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.494 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.495 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.495 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.496 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.496 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.496 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.496 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_default_authority": { 2020-05-24T13:49:14.496 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_default_authority", 2020-05-24T13:49:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "localhost", 2020-05-24T13:49:14.497 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.498 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_password": { 2020-05-24T13:49:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_password", 2020-05-24T13:49:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.499 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.500 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.501 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_port": { 2020-05-24T13:49:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_port", 2020-05-24T13:49:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.502 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "31400", 2020-05-24T13:49:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.503 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_retrieve_prediction_interval": { 2020-05-24T13:49:14.504 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_retrieve_prediction_interval", 2020-05-24T13:49:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.505 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "43200", 2020-05-24T13:49:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.506 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.507 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_server": { 2020-05-24T13:49:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_server", 2020-05-24T13:49:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.508 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.509 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_ssl_target_name_override": { 2020-05-24T13:49:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_ssl_target_name_override", 2020-05-24T13:49:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.510 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.511 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.511 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "localhost", 2020-05-24T13:49:14.511 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.512 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.512 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.512 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.512 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.512 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.513 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.513 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.513 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_upload_metrics_interval": { 2020-05-24T13:49:14.513 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_upload_metrics_interval", 2020-05-24T13:49:14.513 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.513 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.514 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.514 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2020-05-24T13:49:14.514 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.514 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.514 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.515 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.515 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.515 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.515 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.515 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.516 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_upload_smart_interval": { 2020-05-24T13:49:14.516 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_upload_smart_interval", 2020-05-24T13:49:14.516 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.516 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.517 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.517 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "43200", 2020-05-24T13:49:14.517 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.517 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.517 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.518 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.518 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.518 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.518 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.518 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.518 INFO:teuthology.orchestra.run.smithi050.stdout: "diskprediction_user": { 2020-05-24T13:49:14.519 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_user", 2020-05-24T13:49:14.519 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.519 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.519 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.519 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.520 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.520 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.520 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.520 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.520 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.520 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.521 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.521 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.521 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2020-05-24T13:49:14.521 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2020-05-24T13:49:14.521 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.522 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.522 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.522 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2020-05-24T13:49:14.522 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.523 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.523 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.523 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.523 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.523 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.524 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.524 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.524 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.524 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.524 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.524 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "diskprediction_local", 2020-05-24T13:49:14.525 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.525 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.525 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.525 INFO:teuthology.orchestra.run.smithi050.stdout: "predict_interval": { 2020-05-24T13:49:14.525 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "predict_interval", 2020-05-24T13:49:14.526 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.526 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.526 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.526 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "86400", 2020-05-24T13:49:14.527 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.527 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.527 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.527 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.527 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.528 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.528 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.528 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.528 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2020-05-24T13:49:14.528 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2020-05-24T13:49:14.529 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.529 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.529 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.529 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "600", 2020-05-24T13:49:14.529 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.529 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.530 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.530 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.530 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.530 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.530 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.530 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.531 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.531 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.531 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.531 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "influx", 2020-05-24T13:49:14.531 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": false, 2020-05-24T13:49:14.532 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "influxdb python module not found", 2020-05-24T13:49:14.532 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.532 INFO:teuthology.orchestra.run.smithi050.stdout: "batch_size": { 2020-05-24T13:49:14.532 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "batch_size", 2020-05-24T13:49:14.533 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.533 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.533 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.533 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5000", 2020-05-24T13:49:14.533 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.534 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.534 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.534 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.534 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.534 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.534 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.535 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.535 INFO:teuthology.orchestra.run.smithi050.stdout: "database": { 2020-05-24T13:49:14.535 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "database", 2020-05-24T13:49:14.535 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.536 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.536 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.536 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "ceph", 2020-05-24T13:49:14.536 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.537 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.537 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.537 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.537 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.538 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.538 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.538 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.538 INFO:teuthology.orchestra.run.smithi050.stdout: "hostname": { 2020-05-24T13:49:14.538 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "hostname", 2020-05-24T13:49:14.538 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.539 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.539 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.539 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "None", 2020-05-24T13:49:14.539 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.539 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.540 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.540 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.540 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.540 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.540 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.540 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.541 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2020-05-24T13:49:14.541 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2020-05-24T13:49:14.541 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.541 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.541 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.541 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "30", 2020-05-24T13:49:14.542 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.542 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.542 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.542 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.543 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.543 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.543 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.543 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.543 INFO:teuthology.orchestra.run.smithi050.stdout: "password": { 2020-05-24T13:49:14.544 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "password", 2020-05-24T13:49:14.544 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.544 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.544 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.544 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "None", 2020-05-24T13:49:14.544 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.545 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.545 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.545 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.545 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.545 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.546 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.546 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.546 INFO:teuthology.orchestra.run.smithi050.stdout: "port": { 2020-05-24T13:49:14.546 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "port", 2020-05-24T13:49:14.546 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.546 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.547 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.547 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "8086", 2020-05-24T13:49:14.547 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.548 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.548 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.548 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.548 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.548 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.548 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.549 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.549 INFO:teuthology.orchestra.run.smithi050.stdout: "ssl": { 2020-05-24T13:49:14.549 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssl", 2020-05-24T13:49:14.549 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.549 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.550 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.550 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "false", 2020-05-24T13:49:14.550 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.550 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.550 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.550 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.551 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.551 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.551 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.551 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.551 INFO:teuthology.orchestra.run.smithi050.stdout: "threads": { 2020-05-24T13:49:14.552 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "threads", 2020-05-24T13:49:14.552 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.552 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.553 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.553 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2020-05-24T13:49:14.553 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.553 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.553 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.554 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.554 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.554 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.554 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.554 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.554 INFO:teuthology.orchestra.run.smithi050.stdout: "username": { 2020-05-24T13:49:14.555 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "username", 2020-05-24T13:49:14.555 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.555 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.555 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.555 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "None", 2020-05-24T13:49:14.556 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.556 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.556 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.556 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.556 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.557 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.557 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.557 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.557 INFO:teuthology.orchestra.run.smithi050.stdout: "verify_ssl": { 2020-05-24T13:49:14.558 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "verify_ssl", 2020-05-24T13:49:14.558 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.558 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.558 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.558 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "true", 2020-05-24T13:49:14.559 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.559 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.559 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.559 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.559 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.559 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.560 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.560 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.560 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.560 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.560 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.561 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "insights", 2020-05-24T13:49:14.561 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.561 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.561 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": {} 2020-05-24T13:49:14.561 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.561 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.562 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "iostat", 2020-05-24T13:49:14.562 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.562 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.562 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": {} 2020-05-24T13:49:14.562 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.563 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.563 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "localpool", 2020-05-24T13:49:14.563 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.563 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.564 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.564 INFO:teuthology.orchestra.run.smithi050.stdout: "failure_domain": { 2020-05-24T13:49:14.564 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "failure_domain", 2020-05-24T13:49:14.564 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.564 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.564 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.565 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "host", 2020-05-24T13:49:14.565 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.565 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.565 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.565 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "failure domain for any created local pool", 2020-05-24T13:49:14.566 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.566 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.566 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.566 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.566 INFO:teuthology.orchestra.run.smithi050.stdout: "min_size": { 2020-05-24T13:49:14.566 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "min_size", 2020-05-24T13:49:14.567 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.567 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.567 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.567 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.567 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.568 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.568 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.568 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default min_size for any created local pool", 2020-05-24T13:49:14.568 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.569 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.569 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.569 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.569 INFO:teuthology.orchestra.run.smithi050.stdout: "num_rep": { 2020-05-24T13:49:14.569 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "num_rep", 2020-05-24T13:49:14.570 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.570 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.570 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.570 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "3", 2020-05-24T13:49:14.570 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.571 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.571 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.571 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default replica count for any created local pool", 2020-05-24T13:49:14.571 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.571 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.572 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.572 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.572 INFO:teuthology.orchestra.run.smithi050.stdout: "pg_num": { 2020-05-24T13:49:14.572 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_num", 2020-05-24T13:49:14.572 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.572 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.573 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.573 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "128", 2020-05-24T13:49:14.573 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.573 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.574 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.574 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "default pg_num for any created local pool", 2020-05-24T13:49:14.574 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.574 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.574 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.575 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.575 INFO:teuthology.orchestra.run.smithi050.stdout: "prefix": { 2020-05-24T13:49:14.575 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prefix", 2020-05-24T13:49:14.575 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.575 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.575 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.576 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.576 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.576 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.576 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.576 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "name prefix for any created local pool", 2020-05-24T13:49:14.577 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.577 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.577 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.577 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.577 INFO:teuthology.orchestra.run.smithi050.stdout: "subtree": { 2020-05-24T13:49:14.577 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "subtree", 2020-05-24T13:49:14.578 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.578 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.578 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.579 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "rack", 2020-05-24T13:49:14.579 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.579 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.579 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.579 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "CRUSH level for which to create a local pool", 2020-05-24T13:49:14.579 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.580 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.580 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.580 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.580 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.580 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.581 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.581 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator_cli", 2020-05-24T13:49:14.581 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.581 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.581 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.581 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator": { 2020-05-24T13:49:14.582 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "orchestrator", 2020-05-24T13:49:14.582 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.582 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.582 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.582 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.583 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.583 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.583 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.583 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.584 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.584 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.584 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.584 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.584 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.584 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.585 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.585 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "pg_autoscaler", 2020-05-24T13:49:14.585 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.585 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.585 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.586 INFO:teuthology.orchestra.run.smithi050.stdout: "sleep_interval": { 2020-05-24T13:49:14.586 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "sleep_interval", 2020-05-24T13:49:14.586 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.586 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.586 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.586 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2020-05-24T13:49:14.587 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.587 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.587 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.587 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.587 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.588 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.588 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.588 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.588 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.589 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.589 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.589 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "progress", 2020-05-24T13:49:14.589 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.589 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.590 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.590 INFO:teuthology.orchestra.run.smithi050.stdout: "max_completed_events": { 2020-05-24T13:49:14.590 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "max_completed_events", 2020-05-24T13:49:14.590 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.590 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.590 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.591 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "50", 2020-05-24T13:49:14.591 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.591 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.591 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.591 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "number of past completed events to remember", 2020-05-24T13:49:14.592 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.592 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.592 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.592 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.592 INFO:teuthology.orchestra.run.smithi050.stdout: "persist_interval": { 2020-05-24T13:49:14.593 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "persist_interval", 2020-05-24T13:49:14.593 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.593 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.593 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 1, 2020-05-24T13:49:14.594 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "5", 2020-05-24T13:49:14.594 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.594 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.594 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.594 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "how frequently to persist completed events", 2020-05-24T13:49:14.595 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.595 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.595 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.595 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.595 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.596 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.596 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "prometheus", 2020-05-24T13:49:14.596 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.596 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.596 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.597 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools": { 2020-05-24T13:49:14.597 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools", 2020-05-24T13:49:14.597 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.597 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.597 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.598 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.598 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.598 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.598 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.598 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.599 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.599 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.599 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.600 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_stats_pools_refresh_interval": { 2020-05-24T13:49:14.600 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_stats_pools_refresh_interval", 2020-05-24T13:49:14.600 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.600 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.600 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.600 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.601 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.601 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.601 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.601 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.601 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.602 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.602 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.602 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.602 INFO:teuthology.orchestra.run.smithi050.stdout: "scrape_interval": { 2020-05-24T13:49:14.602 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "scrape_interval", 2020-05-24T13:49:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.603 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.604 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.604 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.604 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.605 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.605 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.605 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.606 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2020-05-24T13:49:14.606 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2020-05-24T13:49:14.606 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.606 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.606 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.607 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.607 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.607 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.607 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.607 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.607 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.608 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.608 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.608 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.608 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2020-05-24T13:49:14.608 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2020-05-24T13:49:14.609 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.609 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.609 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.610 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.610 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.610 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.610 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.610 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.611 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.611 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.611 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.611 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.611 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.612 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.612 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.612 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rbd_support", 2020-05-24T13:49:14.612 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.612 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.613 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": {} 2020-05-24T13:49:14.613 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.613 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.613 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "restful", 2020-05-24T13:49:14.613 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.614 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.614 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.614 INFO:teuthology.orchestra.run.smithi050.stdout: "key_file": { 2020-05-24T13:49:14.615 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "key_file", 2020-05-24T13:49:14.615 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.615 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.615 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.615 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.616 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.616 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.616 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.616 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.616 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.617 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.617 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.617 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.617 INFO:teuthology.orchestra.run.smithi050.stdout: "server_addr": { 2020-05-24T13:49:14.617 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_addr", 2020-05-24T13:49:14.618 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.618 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.618 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.618 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.618 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.619 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.619 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.619 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.619 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.620 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.620 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.620 INFO:teuthology.orchestra.run.smithi050.stdout: "server_port": { 2020-05-24T13:49:14.621 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "server_port", 2020-05-24T13:49:14.621 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.621 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.621 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.621 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.622 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.622 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.622 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.622 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.622 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.623 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.623 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.623 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.623 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.624 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.624 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rook", 2020-05-24T13:49:14.624 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.624 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.625 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": {} 2020-05-24T13:49:14.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.625 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.625 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "selftest", 2020-05-24T13:49:14.626 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.626 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.626 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.626 INFO:teuthology.orchestra.run.smithi050.stdout: "roption1": { 2020-05-24T13:49:14.626 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption1", 2020-05-24T13:49:14.627 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.627 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.627 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.627 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.627 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.628 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.628 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.628 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.628 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.628 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.629 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.629 INFO:teuthology.orchestra.run.smithi050.stdout: "roption2": { 2020-05-24T13:49:14.629 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "roption2", 2020-05-24T13:49:14.629 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.630 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.630 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.630 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "xyz", 2020-05-24T13:49:14.631 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.631 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.631 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.631 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.631 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.631 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.632 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.632 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.632 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption1": { 2020-05-24T13:49:14.632 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption1", 2020-05-24T13:49:14.632 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.633 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.633 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.633 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.633 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.633 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.633 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.634 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.634 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.634 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.634 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.635 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption2": { 2020-05-24T13:49:14.635 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption2", 2020-05-24T13:49:14.635 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.636 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.636 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.636 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.636 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.636 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.637 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.637 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.637 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.637 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.637 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.638 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.638 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption3": { 2020-05-24T13:49:14.638 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption3", 2020-05-24T13:49:14.638 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "float", 2020-05-24T13:49:14.639 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.639 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.639 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.639 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.639 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.639 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.640 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.640 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.640 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.641 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.641 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.641 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption4": { 2020-05-24T13:49:14.641 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption4", 2020-05-24T13:49:14.641 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.642 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.642 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.642 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.642 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.642 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.642 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.644 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption5": { 2020-05-24T13:49:14.644 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption5", 2020-05-24T13:49:14.644 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.644 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.644 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.644 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.645 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.645 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.645 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.645 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.646 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.646 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.646 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.646 INFO:teuthology.orchestra.run.smithi050.stdout: "rwoption6": { 2020-05-24T13:49:14.647 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "rwoption6", 2020-05-24T13:49:14.647 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.647 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.647 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.647 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.647 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.648 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.648 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.648 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.648 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.648 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.649 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.649 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.649 INFO:teuthology.orchestra.run.smithi050.stdout: "testkey": { 2020-05-24T13:49:14.649 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testkey", 2020-05-24T13:49:14.649 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.650 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.650 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.650 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.650 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.651 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.651 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.651 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.652 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.652 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.652 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.652 INFO:teuthology.orchestra.run.smithi050.stdout: "testlkey": { 2020-05-24T13:49:14.652 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testlkey", 2020-05-24T13:49:14.653 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.653 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.653 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.653 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.653 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.654 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.654 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.654 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.654 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.654 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.655 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.655 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.655 INFO:teuthology.orchestra.run.smithi050.stdout: "testnewline": { 2020-05-24T13:49:14.656 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "testnewline", 2020-05-24T13:49:14.656 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.656 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.656 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.656 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.657 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.657 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.657 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.657 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.657 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.658 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.658 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.658 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.658 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.658 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.659 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.659 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh", 2020-05-24T13:49:14.659 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.659 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.659 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.660 INFO:teuthology.orchestra.run.smithi050.stdout: "inventory_cache_timeout_min": { 2020-05-24T13:49:14.660 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "inventory_cache_timeout_min", 2020-05-24T13:49:14.660 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.660 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.660 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.661 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.661 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.661 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.662 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.662 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.662 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.662 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.663 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.663 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.663 INFO:teuthology.orchestra.run.smithi050.stdout: "ssh_config_file": { 2020-05-24T13:49:14.663 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "ssh_config_file", 2020-05-24T13:49:14.663 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.664 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.664 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.664 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.664 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.664 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.664 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.665 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.665 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.665 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.665 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.665 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.666 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.666 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.666 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.667 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "status", 2020-05-24T13:49:14.667 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.667 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.667 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": {} 2020-05-24T13:49:14.667 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.668 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.668 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telegraf", 2020-05-24T13:49:14.668 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.668 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.668 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.669 INFO:teuthology.orchestra.run.smithi050.stdout: "address": { 2020-05-24T13:49:14.669 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "address", 2020-05-24T13:49:14.669 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.669 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.669 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.670 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2020-05-24T13:49:14.670 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.670 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.670 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.670 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.671 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.671 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.671 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.671 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.672 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2020-05-24T13:49:14.672 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2020-05-24T13:49:14.672 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.672 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.673 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.673 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "15", 2020-05-24T13:49:14.673 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.673 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.673 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.674 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.674 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.674 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.674 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.674 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.675 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.675 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.675 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.675 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "telemetry", 2020-05-24T13:49:14.675 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.676 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.676 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.676 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_basic": { 2020-05-24T13:49:14.677 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_basic", 2020-05-24T13:49:14.677 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.677 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.677 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.677 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.678 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.678 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.678 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.678 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "Share basic cluster information (size, version)", 2020-05-24T13:49:14.678 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.679 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.679 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.679 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.679 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_crash": { 2020-05-24T13:49:14.679 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_crash", 2020-05-24T13:49:14.679 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.680 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.680 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.680 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.680 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.680 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.681 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.681 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.681 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.681 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.681 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.682 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.682 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_device": { 2020-05-24T13:49:14.682 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_device", 2020-05-24T13:49:14.683 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.683 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.683 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.683 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "True", 2020-05-24T13:49:14.683 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.684 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.684 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.684 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.684 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.684 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.685 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.685 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.685 INFO:teuthology.orchestra.run.smithi050.stdout: "channel_ident": { 2020-05-24T13:49:14.685 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "channel_ident", 2020-05-24T13:49:14.685 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.686 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.686 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.686 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2020-05-24T13:49:14.687 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.687 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.687 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.687 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.687 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.688 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.688 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.688 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.688 INFO:teuthology.orchestra.run.smithi050.stdout: "contact": { 2020-05-24T13:49:14.688 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "contact", 2020-05-24T13:49:14.689 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.689 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.689 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.689 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "None", 2020-05-24T13:49:14.689 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.690 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.690 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.690 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.690 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.690 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.691 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.691 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.691 INFO:teuthology.orchestra.run.smithi050.stdout: "description": { 2020-05-24T13:49:14.691 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "description", 2020-05-24T13:49:14.691 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.692 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.692 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.692 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "None", 2020-05-24T13:49:14.693 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.693 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.693 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.693 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.694 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.694 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.694 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.694 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.694 INFO:teuthology.orchestra.run.smithi050.stdout: "device_url": { 2020-05-24T13:49:14.694 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "device_url", 2020-05-24T13:49:14.695 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.695 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.695 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.695 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/device", 2020-05-24T13:49:14.695 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.695 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.696 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.696 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.696 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.696 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.696 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.697 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.697 INFO:teuthology.orchestra.run.smithi050.stdout: "enabled": { 2020-05-24T13:49:14.697 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "enabled", 2020-05-24T13:49:14.697 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.698 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.698 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.698 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2020-05-24T13:49:14.698 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.698 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.699 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.699 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.699 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.699 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.699 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.699 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.700 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2020-05-24T13:49:14.700 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2020-05-24T13:49:14.700 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.700 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.700 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.701 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "24", 2020-05-24T13:49:14.701 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "8", 2020-05-24T13:49:14.701 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.701 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.701 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.701 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.702 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.702 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.702 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.702 INFO:teuthology.orchestra.run.smithi050.stdout: "last_opt_revision": { 2020-05-24T13:49:14.702 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "last_opt_revision", 2020-05-24T13:49:14.703 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.703 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.703 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.704 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "1", 2020-05-24T13:49:14.704 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.704 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.704 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.704 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.705 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.705 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.705 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.705 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.705 INFO:teuthology.orchestra.run.smithi050.stdout: "leaderboard": { 2020-05-24T13:49:14.705 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "leaderboard", 2020-05-24T13:49:14.706 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "bool", 2020-05-24T13:49:14.706 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.706 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.706 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "False", 2020-05-24T13:49:14.706 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.707 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.707 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.707 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.708 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.708 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.708 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.708 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.708 INFO:teuthology.orchestra.run.smithi050.stdout: "organization": { 2020-05-24T13:49:14.709 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "organization", 2020-05-24T13:49:14.709 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.709 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.709 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.709 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "None", 2020-05-24T13:49:14.710 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.710 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.710 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.710 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.710 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.711 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.711 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.711 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.711 INFO:teuthology.orchestra.run.smithi050.stdout: "proxy": { 2020-05-24T13:49:14.711 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "proxy", 2020-05-24T13:49:14.712 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.712 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.712 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.712 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "None", 2020-05-24T13:49:14.712 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.712 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.713 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.713 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.714 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.714 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.714 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.714 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.714 INFO:teuthology.orchestra.run.smithi050.stdout: "url": { 2020-05-24T13:49:14.715 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "url", 2020-05-24T13:49:14.715 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.715 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.715 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.715 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "https://telemetry.ceph.com/report", 2020-05-24T13:49:14.715 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "test_orchestrator", 2020-05-24T13:49:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": {} 2020-05-24T13:49:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "volumes", 2020-05-24T13:49:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": {} 2020-05-24T13:49:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: { 2020-05-24T13:49:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix", 2020-05-24T13:49:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: "can_run": true, 2020-05-24T13:49:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: "error_string": "", 2020-05-24T13:49:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: "module_options": { 2020-05-24T13:49:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: "identifier": { 2020-05-24T13:49:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "identifier", 2020-05-24T13:49:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "", 2020-05-24T13:49:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: "interval": { 2020-05-24T13:49:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "interval", 2020-05-24T13:49:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "secs", 2020-05-24T13:49:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "60", 2020-05-24T13:49:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_host": { 2020-05-24T13:49:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_host", 2020-05-24T13:49:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "None", 2020-05-24T13:49:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.728 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.728 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.728 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.728 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.728 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.729 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.729 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.729 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_port": { 2020-05-24T13:49:14.729 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_port", 2020-05-24T13:49:14.729 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "int", 2020-05-24T13:49:14.730 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.730 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.730 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "10051", 2020-05-24T13:49:14.730 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.730 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.730 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.731 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.731 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.731 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.731 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.731 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.731 INFO:teuthology.orchestra.run.smithi050.stdout: "zabbix_sender": { 2020-05-24T13:49:14.732 INFO:teuthology.orchestra.run.smithi050.stdout: "name": "zabbix_sender", 2020-05-24T13:49:14.732 INFO:teuthology.orchestra.run.smithi050.stdout: "type": "str", 2020-05-24T13:49:14.732 INFO:teuthology.orchestra.run.smithi050.stdout: "level": "advanced", 2020-05-24T13:49:14.732 INFO:teuthology.orchestra.run.smithi050.stdout: "flags": 0, 2020-05-24T13:49:14.732 INFO:teuthology.orchestra.run.smithi050.stdout: "default_value": "/usr/bin/zabbix_sender", 2020-05-24T13:49:14.733 INFO:teuthology.orchestra.run.smithi050.stdout: "min": "", 2020-05-24T13:49:14.733 INFO:teuthology.orchestra.run.smithi050.stdout: "max": "", 2020-05-24T13:49:14.733 INFO:teuthology.orchestra.run.smithi050.stdout: "enum_allowed": [], 2020-05-24T13:49:14.733 INFO:teuthology.orchestra.run.smithi050.stdout: "desc": "", 2020-05-24T13:49:14.733 INFO:teuthology.orchestra.run.smithi050.stdout: "long_desc": "", 2020-05-24T13:49:14.733 INFO:teuthology.orchestra.run.smithi050.stdout: "tags": [], 2020-05-24T13:49:14.734 INFO:teuthology.orchestra.run.smithi050.stdout: "see_also": [] 2020-05-24T13:49:14.734 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.734 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.735 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.735 INFO:teuthology.orchestra.run.smithi050.stdout: ], 2020-05-24T13:49:14.735 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {}, 2020-05-24T13:49:14.735 INFO:teuthology.orchestra.run.smithi050.stdout: "always_on_modules": { 2020-05-24T13:49:14.735 INFO:teuthology.orchestra.run.smithi050.stdout: "nautilus": [ 2020-05-24T13:49:14.736 INFO:teuthology.orchestra.run.smithi050.stdout: "balancer", 2020-05-24T13:49:14.736 INFO:teuthology.orchestra.run.smithi050.stdout: "crash", 2020-05-24T13:49:14.736 INFO:teuthology.orchestra.run.smithi050.stdout: "devicehealth", 2020-05-24T13:49:14.736 INFO:teuthology.orchestra.run.smithi050.stdout: "orchestrator_cli", 2020-05-24T13:49:14.737 INFO:teuthology.orchestra.run.smithi050.stdout: "progress", 2020-05-24T13:49:14.737 INFO:teuthology.orchestra.run.smithi050.stdout: "rbd_support", 2020-05-24T13:49:14.737 INFO:teuthology.orchestra.run.smithi050.stdout: "status", 2020-05-24T13:49:14.737 INFO:teuthology.orchestra.run.smithi050.stdout: "volumes" 2020-05-24T13:49:14.737 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2020-05-24T13:49:14.738 INFO:teuthology.orchestra.run.smithi050.stdout: } 2020-05-24T13:49:14.738 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.738 INFO:teuthology.orchestra.run.smithi050.stdout: "servicemap": { 2020-05-24T13:49:14.738 INFO:teuthology.orchestra.run.smithi050.stdout: "epoch": 1, 2020-05-24T13:49:14.739 INFO:teuthology.orchestra.run.smithi050.stdout: "modified": "0.000000", 2020-05-24T13:49:14.739 INFO:teuthology.orchestra.run.smithi050.stdout: "services": {} 2020-05-24T13:49:14.739 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2020-05-24T13:49:14.739 INFO:teuthology.orchestra.run.smithi050.stdout: "progress_events": {} 2020-05-24T13:49:14.739 INFO:teuthology.orchestra.run.smithi050.stdout:} 2020-05-24T13:49:14.741 INFO:tasks.ceph.ceph_manager.c1:{'fsid': 'c458a90b-5841-48fd-9e05-3dd82e2be174', 'health': {'checks': {}, 'status': 'HEALTH_OK'}, 'election_epoch': 3, 'quorum': [0], 'quorum_names': ['a'], 'quorum_age': 40, 'monmap': {'epoch': 1, 'fsid': 'c458a90b-5841-48fd-9e05-3dd82e2be174', 'modified': '2020-05-24 13:48:11.108452', 'created': '2020-05-24 13:48:11.108452', 'min_mon_release': 14, 'min_mon_release_name': 'nautilus', 'features': {'persistent': ['kraken', 'luminous', 'mimic', 'osdmap-prune', 'nautilus'], 'optional': []}, 'mons': [{'rank': 0, 'name': 'a', 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:3300', 'nonce': 0}, {'type': 'v1', 'addr': '172.21.15.50:6789', 'nonce': 0}]}, 'addr': '172.21.15.50:6789/0', 'public_addr': '172.21.15.50:6789/0'}]}, 'osdmap': {'osdmap': {'epoch': 13, 'num_osds': 3, 'num_up_osds': 3, 'num_in_osds': 3, 'num_remapped_pgs': 0}}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 8}], 'num_pgs': 8, 'num_pools': 1, 'num_objects': 0, 'data_bytes': 0, 'bytes_used': 3221594112, 'bytes_avail': 318900953088, 'bytes_total': 322122547200}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'epoch': 3, 'active_gid': 4100, 'active_name': 'x', 'active_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6800', 'nonce': 9595}, {'type': 'v1', 'addr': '172.21.15.50:6801', 'nonce': 9595}]}, 'active_addr': '172.21.15.50:6801/9595', 'active_change': '2020-05-24 13:48:38.492669', 'available': True, 'standbys': [], 'modules': ['iostat', 'restful'], 'available_modules': [{'name': 'alerts', 'can_run': True, 'error_string': '', 'module_options': {'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'How frequently to reexamine health status', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_destination': {'name': 'smtp_destination', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email address to send alerts to', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_from_name': {'name': 'smtp_from_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'Ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email From: name', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_host': {'name': 'smtp_host', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_password': {'name': 'smtp_password', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Password to authenticate with', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_port': {'name': 'smtp_port', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '465', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP port', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_sender': {'name': 'smtp_sender', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP envelope sender', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_ssl': {'name': 'smtp_ssl', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Use SSL to connect to SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_user': {'name': 'smtp_user', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'User to authenticate as', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ansible', 'can_run': True, 'error_string': '', 'module_options': {'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_url': {'name': 'server_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_server': {'name': 'verify_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'balancer', 'can_run': True, 'error_string': '', 'module_options': {'active': {'name': 'active', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically balance PGs across cluster', 'long_desc': '', 'tags': [], 'see_also': []}, 'begin_time': {'name': 'begin_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '0000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'beginning time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'begin_weekday': {'name': 'begin_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to this day of the week or later', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'crush_compat_max_iterations': {'name': 'crush_compat_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '25', 'min': '1', 'max': '250', 'enum_allowed': [], 'desc': 'maximum number of iterations to attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'crush_compat_metrics': {'name': 'crush_compat_metrics', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'pgs,objects,bytes', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'metrics with which to calculate OSD utilization', 'long_desc': 'Value is a list of one or more of "pgs", "objects", or "bytes", and indicates which metrics to use to balance utilization.', 'tags': [], 'see_also': []}, 'crush_compat_step': {'name': 'crush_compat_step', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0.5', 'min': '0.001', 'max': '0.999', 'enum_allowed': [], 'desc': 'aggressiveness of optimization', 'long_desc': '.99 is very aggressive, .01 is less aggressive', 'tags': [], 'see_also': []}, 'end_time': {'name': 'end_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '2400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'ending time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'end_weekday': {'name': 'end_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '7', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to days of the week earlier than this', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'min_score': {'name': 'min_score', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'minimum score, below which no optimization is attempted', 'long_desc': '', 'tags': [], 'see_also': []}, 'mode': {'name': 'mode', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'none', 'min': '', 'max': '', 'enum_allowed': ['crush-compat', 'none', 'upmap'], 'desc': 'Balancer mode', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_ids': {'name': 'pool_ids', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'pools which the automatic balancing will be limited to', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'upmap_max_deviation': {'name': 'upmap_max_deviation', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '1', 'max': '', 'enum_allowed': [], 'desc': 'deviation below which no optimization is attempted', 'long_desc': 'If the number of PGs are within this count then no optimization is attempted', 'tags': [], 'see_also': []}, 'upmap_max_iterations': {'name': 'upmap_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '10', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'maximum upmap optimization iterations', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'crash', 'can_run': True, 'error_string': '', 'module_options': {'retain_interval': {'name': 'retain_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '31536000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain crashes before pruning them', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_recent_interval': {'name': 'warn_recent_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '1209600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'time interval in which to warn about recent crashes', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'dashboard', 'can_run': True, 'error_string': '', 'module_options': {'ALERTMANAGER_API_HOST': {'name': 'ALERTMANAGER_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_ENABLED': {'name': 'AUDIT_API_ENABLED', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_LOG_PAYLOAD': {'name': 'AUDIT_API_LOG_PAYLOAD', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ENABLE_BROWSABLE_API': {'name': 'ENABLE_BROWSABLE_API', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_cephfs': {'name': 'FEATURE_TOGGLE_cephfs', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_iscsi': {'name': 'FEATURE_TOGGLE_iscsi', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_mirroring': {'name': 'FEATURE_TOGGLE_mirroring', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rbd': {'name': 'FEATURE_TOGGLE_rbd', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rgw': {'name': 'FEATURE_TOGGLE_rgw', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE': {'name': 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_PASSWORD': {'name': 'GRAFANA_API_PASSWORD', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_SSL_VERIFY': {'name': 'GRAFANA_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_URL': {'name': 'GRAFANA_API_URL', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_USERNAME': {'name': 'GRAFANA_API_USERNAME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_UPDATE_DASHBOARDS': {'name': 'GRAFANA_UPDATE_DASHBOARDS', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ISCSI_API_SSL_VERIFICATION': {'name': 'ISCSI_API_SSL_VERIFICATION', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'PROMETHEUS_API_HOST': {'name': 'PROMETHEUS_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'REST_REQUESTS_TIMEOUT': {'name': 'REST_REQUESTS_TIMEOUT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '45', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ACCESS_KEY': {'name': 'RGW_API_ACCESS_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ADMIN_RESOURCE': {'name': 'RGW_API_ADMIN_RESOURCE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_HOST': {'name': 'RGW_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_PORT': {'name': 'RGW_API_PORT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '80', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SCHEME': {'name': 'RGW_API_SCHEME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'http', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SECRET_KEY': {'name': 'RGW_API_SECRET_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SSL_VERIFY': {'name': 'RGW_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_USER_ID': {'name': 'RGW_API_USER_ID', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'crt_file': {'name': 'crt_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'debug': {'name': 'debug', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Enable/disable debug options', 'long_desc': '', 'tags': [], 'see_also': []}, 'jwt_token_ttl': {'name': 'jwt_token_ttl', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '28800', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '::', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8080', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl_server_port': {'name': 'ssl_server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8443', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_behaviour': {'name': 'standby_behaviour', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'redirect', 'min': '', 'max': '', 'enum_allowed': ['error', 'redirect'], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_error_status_code': {'name': 'standby_error_status_code', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '500', 'min': '400', 'max': '599', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url_prefix': {'name': 'url_prefix', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'deepsea', 'can_run': True, 'error_string': '', 'module_options': {'salt_api_eauth': {'name': 'salt_api_eauth', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'sharedsecret', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_password': {'name': 'salt_api_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_url': {'name': 'salt_api_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_username': {'name': 'salt_api_username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'devicehealth', 'can_run': True, 'error_string': '', 'module_options': {'enable_monitoring': {'name': 'enable_monitoring', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'monitor device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'mark_out_threshold': {'name': 'mark_out_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '2419200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically mark OSD if it may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_name': {'name': 'pool_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'device_health_metrics', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name of pool in which to store device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'retention_period': {'name': 'retention_period', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '15552000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_frequency': {'name': 'scrape_frequency', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to scrape device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'self_heal': {'name': 'self_heal', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'preemptively heal cluster around devices that may fail', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and check device health', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_threshold': {'name': 'warn_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '7257600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'raise health warning if OSD may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_cloud', 'can_run': True, 'error_string': '', 'module_options': {'diskprediction_cert_context': {'name': 'diskprediction_cert_context', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_default_authority': {'name': 'diskprediction_default_authority', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_password': {'name': 'diskprediction_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_port': {'name': 'diskprediction_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '31400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_retrieve_prediction_interval': {'name': 'diskprediction_retrieve_prediction_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_server': {'name': 'diskprediction_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_ssl_target_name_override': {'name': 'diskprediction_ssl_target_name_override', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_metrics_interval': {'name': 'diskprediction_upload_metrics_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_smart_interval': {'name': 'diskprediction_upload_smart_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_user': {'name': 'diskprediction_user', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_local', 'can_run': True, 'error_string': '', 'module_options': {'predict_interval': {'name': 'predict_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'influx', 'can_run': False, 'error_string': 'influxdb python module not found', 'module_options': {'batch_size': {'name': 'batch_size', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'database': {'name': 'database', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'hostname': {'name': 'hostname', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '30', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'port': {'name': 'port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '8086', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'false', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'threads': {'name': 'threads', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_ssl': {'name': 'verify_ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'true', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'insights', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'iostat', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'localpool', 'can_run': True, 'error_string': '', 'module_options': {'failure_domain': {'name': 'failure_domain', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'host', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'failure domain for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'min_size': {'name': 'min_size', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default min_size for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'num_rep': {'name': 'num_rep', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '3', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default replica count for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'pg_num': {'name': 'pg_num', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '128', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default pg_num for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'prefix': {'name': 'prefix', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name prefix for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'subtree': {'name': 'subtree', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'rack', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'CRUSH level for which to create a local pool', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'orchestrator_cli', 'can_run': True, 'error_string': '', 'module_options': {'orchestrator': {'name': 'orchestrator', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'pg_autoscaler', 'can_run': True, 'error_string': '', 'module_options': {'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'progress', 'can_run': True, 'error_string': '', 'module_options': {'max_completed_events': {'name': 'max_completed_events', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '50', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'number of past completed events to remember', 'long_desc': '', 'tags': [], 'see_also': []}, 'persist_interval': {'name': 'persist_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to persist completed events', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'prometheus', 'can_run': True, 'error_string': '', 'module_options': {'rbd_stats_pools': {'name': 'rbd_stats_pools', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rbd_stats_pools_refresh_interval': {'name': 'rbd_stats_pools_refresh_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_interval': {'name': 'scrape_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rbd_support', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'restful', 'can_run': True, 'error_string': '', 'module_options': {'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rook', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'selftest', 'can_run': True, 'error_string': '', 'module_options': {'roption1': {'name': 'roption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'roption2': {'name': 'roption2', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'xyz', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption1': {'name': 'rwoption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption2': {'name': 'rwoption2', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption3': {'name': 'rwoption3', 'type': 'float', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption4': {'name': 'rwoption4', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption5': {'name': 'rwoption5', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption6': {'name': 'rwoption6', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testkey': {'name': 'testkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testlkey': {'name': 'testlkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testnewline': {'name': 'testnewline', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ssh', 'can_run': True, 'error_string': '', 'module_options': {'inventory_cache_timeout_min': {'name': 'inventory_cache_timeout_min', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssh_config_file': {'name': 'ssh_config_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'status', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'telegraf', 'can_run': True, 'error_string': '', 'module_options': {'address': {'name': 'address', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'unixgram:///tmp/telegraf.sock', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '15', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'telemetry', 'can_run': True, 'error_string': '', 'module_options': {'channel_basic': {'name': 'channel_basic', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Share basic cluster information (size, version)', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_crash': {'name': 'channel_crash', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_device': {'name': 'channel_device', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_ident': {'name': 'channel_ident', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'contact': {'name': 'contact', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'description': {'name': 'description', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'device_url': {'name': 'device_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/device', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'enabled': {'name': 'enabled', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '24', 'min': '8', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'last_opt_revision': {'name': 'last_opt_revision', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '1', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'leaderboard': {'name': 'leaderboard', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'organization': {'name': 'organization', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'proxy': {'name': 'proxy', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url': {'name': 'url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/report', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'test_orchestrator', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'volumes', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'zabbix', 'can_run': True, 'error_string': '', 'module_options': {'identifier': {'name': 'identifier', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_host': {'name': 'zabbix_host', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_port': {'name': 'zabbix_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '10051', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_sender': {'name': 'zabbix_sender', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '/usr/bin/zabbix_sender', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}], 'services': {}, 'always_on_modules': {'nautilus': ['balancer', 'crash', 'devicehealth', 'orchestrator_cli', 'progress', 'rbd_support', 'status', 'volumes']}}, 'servicemap': {'epoch': 1, 'modified': '0.000000', 'services': {}}, 'progress_events': {}} 2020-05-24T13:49:14.744 INFO:tasks.ceph.ceph_manager.c1:clean! 2020-05-24T13:49:14.745 INFO:tasks.ceph:Waiting until ceph cluster c1 is healthy... 2020-05-24T13:49:14.745 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 health 2020-05-24T13:49:15.758 INFO:teuthology.misc.health.smithi050.stdout:HEALTH_OK 2020-05-24T13:49:15.780 DEBUG:teuthology.misc:Ceph health: HEALTH_OK 2020-05-24T13:49:15.781 INFO:teuthology.run_tasks:Running task ceph... 2020-05-24T13:49:15.791 INFO:tasks.ceph:Creating ceph cluster c2... 2020-05-24T13:49:15.792 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'c2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-05-24T13:49:15.792 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi/5087414', 'branch': 'nautilus', 'description': 'rgw/multisite/{clusters.yaml frontend/civetweb.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml}', 'email': 'tchaikov@gmail.com', 'first_in_suite': False, 'job_id': '5087414', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'kchai-2020-05-24_13:25:07-rgw-nautilus-distro-basic-smithi', 'nuke-on-error': True, 'os_type': 'centos', 'overrides': {'admin_socket': {'branch': 'nautilus'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078'}}, 'rgw': {'compression type': 'random', 'frontend': 'civetweb'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}, {'endpoints': ['c1.client.1'], 'name': 'test-zone3'}, {'endpoints': ['c2.client.1'], 'is_pubsub': True, 'name': 'test-zone4'}]}]}, 'workunit': {'branch': 'nautilus-py3', 'sha1': '2e1d102f0cb696365106419cf1e42700ba50a8ff'}}, 'owner': 'scheduled_kchai@teuthology', 'priority': 90, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'], ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'nautilus-py3', 'suite_path': '/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/tchaikov/ceph.git', 'suite_sha1': '2e1d102f0cb696365106419cf1e42700ba50a8ff', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}}, {'ceph': {'cluster': 'c2', 'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync log trim interval': 0}, 'global': {'osd heartbeat grace': 40}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'osd fast shutdown': False}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6fe217b2dddadf1baae6a015a57f83f4283a6078', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'wait-for-scrub': False}}, {'rgw': {'c1.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c1.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 60}}}], 'teuthology_branch': 'master', 'tube': 'smithi', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.30709', 'targets': {'smithi050.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0YXVafo/xd3CdpgUWNATDXI81GKJ0ZICl+CwvHbakJEQbr4WNtjRDwwJfd6XQoW6raB3VIgqfoaPfxPNVsKHwentf6qQ3k3a9713qUVRkm/qJ0op2SHVSl1j6I9kpB66p1Qjlun9XbSOprdByFbBEIgbcSGw1S2W4YKJIJnAYhSR/gQZQJPqH2giCCJv34D63vfh4BvExEygU8Y0IITtnqPwLFugXEFfLAeCgZ7/LDiWtpaHOTbVgfiO320qWw5dx7l2CEiqxmJdjJrcS67+2TQY2kicRH8qWiHhWh3JueRZldMXP1X3P7eqPY75SK8M5uYY8Guc5frCm1TcMSMlH', 'smithi160.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbjOIgGz2r4F2rEiE18SEix1zO1uILNCLKpnyWJ3HF/upvHd69PmvXN8b1Mw3h417K/U2qzhX+yBYW9GZpuDcSi6fAoI2ripRRrYZX4yj3NyYuflpwlfxCIR/WHOH3Q5hdHfRy3L9GeA6/M7uTsHu8lFv1+bzh1pkekIr97BNdu8SkZO4Jg/VCyA+iGX/Y/YkxSJkJ1ZeE2FQgXslTKRNRBzvCoheXa1nvETZL0EdJr1PqP8RQwpK5LCnuMutlWJYU8nvf4wTLhcv/7TpJyohyK8CCAMqMRSBUAt6jho5DfLfPm/HeCwzOPkZp3/qAr5cQCNujg7WMmgl2zl556cX3'}} 2020-05-24T13:49:15.794 INFO:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2020-05-24T13:49:15.798 INFO:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2020-05-24T13:49:15.889 INFO:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/run/ceph 2020-05-24T13:49:15.897 INFO:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/run/ceph 2020-05-24T13:49:16.090 DEBUG:teuthology.orchestra.remote:smithi160:/scratch_devs is 72B 2020-05-24T13:49:16.169 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-05-24T13:49:16.170 INFO:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-05-24T13:49:16.238 INFO:teuthology.orchestra.run.smithi160.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-0’ 2020-05-24T13:49:16.239 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:49:16.239 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 106023 Links: 1 2020-05-24T13:49:16.239 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:49:16.239 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:49:16.239 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2020-05-24 13:47:03.500769738 +0000 2020-05-24T13:49:16.240 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2020-05-24 13:45:21.571765766 +0000 2020-05-24T13:49:16.240 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2020-05-24 13:45:21.571765766 +0000 2020-05-24T13:49:16.240 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2020-05-24T13:49:16.261 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2020-05-24T13:49:16.262 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2020-05-24T13:49:16.262 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes (512 B) copied, 0.00034106 s, 1.5 MB/s 2020-05-24T13:49:16.265 INFO:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-05-24T13:49:16.387 INFO:teuthology.orchestra.run.smithi160.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-1’ 2020-05-24T13:49:16.388 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:49:16.388 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 110882 Links: 1 2020-05-24T13:49:16.388 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:49:16.388 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:49:16.388 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2020-05-24 13:47:03.500769738 +0000 2020-05-24T13:49:16.389 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2020-05-24 13:45:22.084758193 +0000 2020-05-24T13:49:16.389 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2020-05-24 13:45:22.084758193 +0000 2020-05-24T13:49:16.389 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2020-05-24T13:49:16.414 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2020-05-24T13:49:16.414 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2020-05-24T13:49:16.415 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes (512 B) copied, 0.000355467 s, 1.4 MB/s 2020-05-24T13:49:16.418 INFO:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-05-24T13:49:16.484 INFO:teuthology.orchestra.run.smithi160.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2020-05-24T13:49:16.484 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:49:16.484 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 113703 Links: 1 2020-05-24T13:49:16.484 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:49:16.485 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:49:16.485 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2020-05-24 13:47:03.500769738 +0000 2020-05-24T13:49:16.485 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2020-05-24 13:45:22.550751317 +0000 2020-05-24T13:49:16.485 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2020-05-24 13:45:22.550751317 +0000 2020-05-24T13:49:16.485 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2020-05-24T13:49:16.510 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2020-05-24T13:49:16.510 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2020-05-24T13:49:16.510 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes (512 B) copied, 0.000432854 s, 1.2 MB/s 2020-05-24T13:49:16.514 INFO:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-05-24T13:49:16.587 INFO:teuthology.orchestra.run.smithi160.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-3’ 2020-05-24T13:49:16.588 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-05-24T13:49:16.588 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 112034 Links: 1 2020-05-24T13:49:16.588 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-05-24T13:49:16.588 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2020-05-24T13:49:16.588 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2020-05-24 13:47:03.500769738 +0000 2020-05-24T13:49:16.589 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2020-05-24 13:45:23.022744351 +0000 2020-05-24T13:49:16.589 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2020-05-24 13:45:23.022744351 +0000 2020-05-24T13:49:16.589 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2020-05-24T13:49:16.610 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2020-05-24T13:49:16.610 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2020-05-24T13:49:16.610 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes (512 B) copied, 0.000331732 s, 1.5 MB/s 2020-05-24T13:49:16.614 INFO:tasks.ceph:osd dev map: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:49:16.614 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}} 2020-05-24T13:49:16.615 INFO:tasks.ceph:Generating config... 2020-05-24T13:49:16.616 INFO:tasks.ceph:[client] debug rgw = 20 2020-05-24T13:49:16.616 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2020-05-24T13:49:16.617 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:49:16.617 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2020-05-24T13:49:16.618 INFO:tasks.ceph:[client] rgw data log num shards = 4 2020-05-24T13:49:16.618 INFO:tasks.ceph:[client] rgw md log max shards = 4 2020-05-24T13:49:16.618 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2020-05-24T13:49:16.618 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2020-05-24T13:49:16.619 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-05-24T13:49:16.619 INFO:tasks.ceph:[mgr] debug ms = 1 2020-05-24T13:49:16.620 INFO:tasks.ceph:[mon] debug mon = 20 2020-05-24T13:49:16.620 INFO:tasks.ceph:[mon] debug ms = 1 2020-05-24T13:49:16.620 INFO:tasks.ceph:[mon] debug paxos = 20 2020-05-24T13:49:16.620 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2020-05-24T13:49:16.621 INFO:tasks.ceph:[osd] debug filestore = 20 2020-05-24T13:49:16.621 INFO:tasks.ceph:[osd] debug journal = 20 2020-05-24T13:49:16.621 INFO:tasks.ceph:[osd] debug ms = 20 2020-05-24T13:49:16.621 INFO:tasks.ceph:[osd] debug osd = 25 2020-05-24T13:49:16.622 INFO:tasks.ceph:[osd] osd fast shutdown = False 2020-05-24T13:49:16.622 INFO:tasks.ceph:[osd.0] osd_max_omap_entries_per_request = 10 2020-05-24T13:49:16.622 INFO:tasks.ceph:[osd.1] osd_max_omap_entries_per_request = 1000 2020-05-24T13:49:16.623 INFO:tasks.ceph:[osd.2] osd_max_omap_entries_per_request = 10000 2020-05-24T13:49:16.623 INFO:tasks.ceph:Setting up c2.mon.a... 2020-05-24T13:49:16.623 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c2.keyring 2020-05-24T13:49:16.777 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/c2.keyring 2020-05-24T13:49:16.781 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c2.keyring 2020-05-24T13:49:16.939 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/c2.keyring 2020-05-24T13:49:17.022 DEBUG:tasks.ceph:Ceph mon addresses: [('c2.mon.a', '172.21.15.160')] 2020-05-24T13:49:17.023 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.160 --print /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:17.139 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:17.139 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool: generated fsid e6d3bc87-1c02-4a97-acf9-0c849318394c 2020-05-24T13:49:17.139 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 0 2020-05-24T13:49:17.140 INFO:teuthology.orchestra.run.smithi160.stdout:fsid e6d3bc87-1c02-4a97-acf9-0c849318394c 2020-05-24T13:49:17.140 INFO:teuthology.orchestra.run.smithi160.stdout:last_changed 2020-05-24 13:49:17.140145 2020-05-24T13:49:17.140 INFO:teuthology.orchestra.run.smithi160.stdout:created 2020-05-24 13:49:17.140145 2020-05-24T13:49:17.140 INFO:teuthology.orchestra.run.smithi160.stdout:min_mon_release 0 (unknown) 2020-05-24T13:49:17.140 INFO:teuthology.orchestra.run.smithi160.stdout:0: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.a 2020-05-24T13:49:17.141 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c2.monmap (1 monitors) 2020-05-24T13:49:17.142 INFO:tasks.ceph:Writing /etc/ceph/c2.conf for FSID e6d3bc87-1c02-4a97-acf9-0c849318394c... 2020-05-24T13:49:17.143 INFO:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2020-05-24T13:49:17.146 INFO:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2020-05-24T13:49:17.311 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2020-05-24T13:49:17.311 INFO:teuthology.orchestra.run.smithi160.stdout: chdir = "" 2020-05-24T13:49:17.311 INFO:teuthology.orchestra.run.smithi160.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-24T13:49:17.312 INFO:teuthology.orchestra.run.smithi160.stdout: auth supported = cephx 2020-05-24T13:49:17.312 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.312 INFO:teuthology.orchestra.run.smithi160.stdout: filestore xattr use omap = true 2020-05-24T13:49:17.312 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.312 INFO:teuthology.orchestra.run.smithi160.stdout: mon clock drift allowed = 1.000 2020-05-24T13:49:17.313 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.313 INFO:teuthology.orchestra.run.smithi160.stdout: osd crush chooseleaf type = 0 2020-05-24T13:49:17.313 INFO:teuthology.orchestra.run.smithi160.stdout: auth debug = true 2020-05-24T13:49:17.313 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.313 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on old message = true 2020-05-24T13:49:17.314 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on bug = true 2020-05-24T13:49:17.314 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.314 INFO:teuthology.orchestra.run.smithi160.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-24T13:49:17.314 INFO:teuthology.orchestra.run.smithi160.stdout: mon pg warn max object skew = 0 2020-05-24T13:49:17.314 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.315 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default size = 2 2020-05-24T13:49:17.315 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.315 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow primary affinity = true 2020-05-24T13:49:17.315 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow pg remap = true 2020-05-24T13:49:17.315 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on legacy crush tunables = false 2020-05-24T13:49:17.315 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on crush straw calc version zero = false 2020-05-24T13:49:17.316 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on no sortbitwise = false 2020-05-24T13:49:17.316 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on osd down out interval zero = false 2020-05-24T13:49:17.316 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on too few osds = false 2020-05-24T13:49:17.316 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-24T13:49:17.316 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-24T13:49:17.316 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.317 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-24T13:49:17.317 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.317 INFO:teuthology.orchestra.run.smithi160.stdout: osd default data pool replay window = 5 2020-05-24T13:49:17.317 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.317 INFO:teuthology.orchestra.run.smithi160.stdout: mon allow pool delete = true 2020-05-24T13:49:17.318 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.318 INFO:teuthology.orchestra.run.smithi160.stdout: mon cluster log file level = debug 2020-05-24T13:49:17.318 INFO:teuthology.orchestra.run.smithi160.stdout: debug asserts on shutdown = true 2020-05-24T13:49:17.318 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.318 INFO:teuthology.orchestra.run.smithi160.stdout: # we see this fail in qa on *nautilus*; bump up retries 2020-05-24T13:49:17.318 INFO:teuthology.orchestra.run.smithi160.stdout: mon_client_directed_command_retry = 4 2020-05-24T13:49:17.319 INFO:teuthology.orchestra.run.smithi160.stdout: mon host = 172.21.15.160 2020-05-24T13:49:17.319 INFO:teuthology.orchestra.run.smithi160.stdout: osd heartbeat grace = 40 2020-05-24T13:49:17.319 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = e6d3bc87-1c02-4a97-acf9-0c849318394c 2020-05-24T13:49:17.319 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.319 INFO:teuthology.orchestra.run.smithi160.stdout:[osd] 2020-05-24T13:49:17.320 INFO:teuthology.orchestra.run.smithi160.stdout: osd journal size = 100 2020-05-24T13:49:17.320 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.320 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub load threshold = 5.0 2020-05-24T13:49:17.320 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub max interval = 600 2020-05-24T13:49:17.320 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.320 INFO:teuthology.orchestra.run.smithi160.stdout: osd recover clone overlap = true 2020-05-24T13:49:17.321 INFO:teuthology.orchestra.run.smithi160.stdout: osd recovery max chunk = 1048576 2020-05-24T13:49:17.321 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.321 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug shutdown = true 2020-05-24T13:49:17.321 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug op order = true 2020-05-24T13:49:17.321 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug verify stray on activate = true 2020-05-24T13:49:17.322 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.322 INFO:teuthology.orchestra.run.smithi160.stdout: osd open classes on start = true 2020-05-24T13:49:17.322 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug pg log writeout = true 2020-05-24T13:49:17.322 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.322 INFO:teuthology.orchestra.run.smithi160.stdout: osd deep scrub update digest min age = 30 2020-05-24T13:49:17.322 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.323 INFO:teuthology.orchestra.run.smithi160.stdout: osd map max advance = 10 2020-05-24T13:49:17.323 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.323 INFO:teuthology.orchestra.run.smithi160.stdout: journal zero on create = true 2020-05-24T13:49:17.323 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.323 INFO:teuthology.orchestra.run.smithi160.stdout: filestore ondisk finisher threads = 3 2020-05-24T13:49:17.324 INFO:teuthology.orchestra.run.smithi160.stdout: filestore apply finisher threads = 3 2020-05-24T13:49:17.324 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.324 INFO:teuthology.orchestra.run.smithi160.stdout: bdev debug aio = true 2020-05-24T13:49:17.324 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug misdirected ops = true 2020-05-24T13:49:17.324 INFO:teuthology.orchestra.run.smithi160.stdout: debug filestore = 20 2020-05-24T13:49:17.324 INFO:teuthology.orchestra.run.smithi160.stdout: debug journal = 20 2020-05-24T13:49:17.325 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 20 2020-05-24T13:49:17.325 INFO:teuthology.orchestra.run.smithi160.stdout: debug osd = 25 2020-05-24T13:49:17.325 INFO:teuthology.orchestra.run.smithi160.stdout: osd fast shutdown = False 2020-05-24T13:49:17.325 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.325 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr] 2020-05-24T13:49:17.326 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2020-05-24T13:49:17.326 INFO:teuthology.orchestra.run.smithi160.stdout: debug mgr = 20 2020-05-24T13:49:17.326 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2020-05-24T13:49:17.326 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2020-05-24T13:49:17.326 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:49:17.326 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:49:17.327 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.327 INFO:teuthology.orchestra.run.smithi160.stdout:[mon] 2020-05-24T13:49:17.327 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2020-05-24T13:49:17.327 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2020-05-24T13:49:17.327 INFO:teuthology.orchestra.run.smithi160.stdout: debug paxos = 20 2020-05-24T13:49:17.327 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2020-05-24T13:49:17.328 INFO:teuthology.orchestra.run.smithi160.stdout: mon data avail warn = 5 2020-05-24T13:49:17.328 INFO:teuthology.orchestra.run.smithi160.stdout: mon mgr mkfs grace = 240 2020-05-24T13:49:17.328 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:49:17.328 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd reporter subtree level = osd 2020-05-24T13:49:17.328 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd prime pg temp = true 2020-05-24T13:49:17.329 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:49:17.329 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd crush smoke test = False 2020-05-24T13:49:17.329 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:49:17.329 INFO:teuthology.orchestra.run.smithi160.stdout:[client] 2020-05-24T13:49:17.329 INFO:teuthology.orchestra.run.smithi160.stdout: rgw cache enabled = true 2020-05-24T13:49:17.329 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable ops log = true 2020-05-24T13:49:17.330 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable usage log = true 2020-05-24T13:49:17.330 INFO:teuthology.orchestra.run.smithi160.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-24T13:49:17.330 INFO:teuthology.orchestra.run.smithi160.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-24T13:49:17.330 INFO:teuthology.orchestra.run.smithi160.stdout: debug rgw = 20 2020-05-24T13:49:17.330 INFO:teuthology.orchestra.run.smithi160.stdout: rgw crypt require ssl = False 2020-05-24T13:49:17.331 INFO:teuthology.orchestra.run.smithi160.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:49:17.331 INFO:teuthology.orchestra.run.smithi160.stdout: rgw curl low speed time = 300 2020-05-24T13:49:17.331 INFO:teuthology.orchestra.run.smithi160.stdout: rgw data log num shards = 4 2020-05-24T13:49:17.331 INFO:teuthology.orchestra.run.smithi160.stdout: rgw md log max shards = 4 2020-05-24T13:49:17.331 INFO:teuthology.orchestra.run.smithi160.stdout: rgw sync log trim interval = 0 2020-05-24T13:49:17.331 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.a] 2020-05-24T13:49:17.332 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.0] 2020-05-24T13:49:17.332 INFO:teuthology.orchestra.run.smithi160.stdout: osd_max_omap_entries_per_request = 10 2020-05-24T13:49:17.332 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.1] 2020-05-24T13:49:17.332 INFO:teuthology.orchestra.run.smithi160.stdout: osd_max_omap_entries_per_request = 1000 2020-05-24T13:49:17.332 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.2] 2020-05-24T13:49:17.332 INFO:teuthology.orchestra.run.smithi160.stdout: osd_max_omap_entries_per_request = 10000 2020-05-24T13:49:17.333 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2020-05-24T13:49:17.334 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2020-05-24T13:49:17.334 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-05-24T13:49:17.334 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2020-05-24T13:49:17.334 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.334 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2020-05-24T13:49:17.335 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.335 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2020-05-24T13:49:17.335 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.335 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2020-05-24T13:49:17.335 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2020-05-24T13:49:17.335 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.336 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2020-05-24T13:49:17.336 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2020-05-24T13:49:17.336 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.336 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2020-05-24T13:49:17.336 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2020-05-24T13:49:17.337 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.337 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2020-05-24T13:49:17.337 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.337 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2020-05-24T13:49:17.337 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2020-05-24T13:49:17.337 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2020-05-24T13:49:17.338 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2020-05-24T13:49:17.338 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2020-05-24T13:49:17.338 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2020-05-24T13:49:17.338 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2020-05-24T13:49:17.338 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-05-24T13:49:17.338 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2020-05-24T13:49:17.339 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.339 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-05-24T13:49:17.339 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.339 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2020-05-24T13:49:17.339 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.340 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2020-05-24T13:49:17.340 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.340 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2020-05-24T13:49:17.340 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2020-05-24T13:49:17.340 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.340 INFO:teuthology.orchestra.run.smithi050.stdout: # we see this fail in qa on *nautilus*; bump up retries 2020-05-24T13:49:17.341 INFO:teuthology.orchestra.run.smithi050.stdout: mon_client_directed_command_retry = 4 2020-05-24T13:49:17.341 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = 172.21.15.160 2020-05-24T13:49:17.341 INFO:teuthology.orchestra.run.smithi050.stdout: osd heartbeat grace = 40 2020-05-24T13:49:17.341 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = e6d3bc87-1c02-4a97-acf9-0c849318394c 2020-05-24T13:49:17.341 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.342 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2020-05-24T13:49:17.342 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2020-05-24T13:49:17.342 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.342 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2020-05-24T13:49:17.342 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 600 2020-05-24T13:49:17.342 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.343 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2020-05-24T13:49:17.343 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2020-05-24T13:49:17.343 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.343 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2020-05-24T13:49:17.343 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2020-05-24T13:49:17.343 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2020-05-24T13:49:17.344 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.344 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2020-05-24T13:49:17.344 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2020-05-24T13:49:17.344 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.344 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2020-05-24T13:49:17.345 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.345 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2020-05-24T13:49:17.345 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.345 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2020-05-24T13:49:17.345 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.345 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2020-05-24T13:49:17.346 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2020-05-24T13:49:17.346 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.346 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2020-05-24T13:49:17.346 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2020-05-24T13:49:17.346 INFO:teuthology.orchestra.run.smithi050.stdout: debug filestore = 20 2020-05-24T13:49:17.346 INFO:teuthology.orchestra.run.smithi050.stdout: debug journal = 20 2020-05-24T13:49:17.347 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 20 2020-05-24T13:49:17.347 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 25 2020-05-24T13:49:17.347 INFO:teuthology.orchestra.run.smithi050.stdout: osd fast shutdown = False 2020-05-24T13:49:17.347 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.347 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2020-05-24T13:49:17.348 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2020-05-24T13:49:17.348 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2020-05-24T13:49:17.348 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2020-05-24T13:49:17.348 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2020-05-24T13:49:17.348 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:49:17.348 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:49:17.349 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.349 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2020-05-24T13:49:17.349 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2020-05-24T13:49:17.349 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2020-05-24T13:49:17.349 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2020-05-24T13:49:17.350 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2020-05-24T13:49:17.350 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2020-05-24T13:49:17.350 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2020-05-24T13:49:17.350 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2020-05-24T13:49:17.350 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2020-05-24T13:49:17.350 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2020-05-24T13:49:17.351 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2020-05-24T13:49:17.351 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd crush smoke test = False 2020-05-24T13:49:17.351 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-05-24T13:49:17.351 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2020-05-24T13:49:17.351 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2020-05-24T13:49:17.351 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2020-05-24T13:49:17.352 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2020-05-24T13:49:17.352 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-05-24T13:49:17.352 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-05-24T13:49:17.352 INFO:teuthology.orchestra.run.smithi050.stdout: debug rgw = 20 2020-05-24T13:49:17.352 INFO:teuthology.orchestra.run.smithi050.stdout: rgw crypt require ssl = False 2020-05-24T13:49:17.353 INFO:teuthology.orchestra.run.smithi050.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2020-05-24T13:49:17.353 INFO:teuthology.orchestra.run.smithi050.stdout: rgw curl low speed time = 300 2020-05-24T13:49:17.353 INFO:teuthology.orchestra.run.smithi050.stdout: rgw data log num shards = 4 2020-05-24T13:49:17.353 INFO:teuthology.orchestra.run.smithi050.stdout: rgw md log max shards = 4 2020-05-24T13:49:17.353 INFO:teuthology.orchestra.run.smithi050.stdout: rgw sync log trim interval = 0 2020-05-24T13:49:17.353 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2020-05-24T13:49:17.354 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.0] 2020-05-24T13:49:17.354 INFO:teuthology.orchestra.run.smithi050.stdout: osd_max_omap_entries_per_request = 10 2020-05-24T13:49:17.354 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.1] 2020-05-24T13:49:17.354 INFO:teuthology.orchestra.run.smithi050.stdout: osd_max_omap_entries_per_request = 1000 2020-05-24T13:49:17.354 INFO:teuthology.orchestra.run.smithi050.stdout:[osd.2] 2020-05-24T13:49:17.354 INFO:teuthology.orchestra.run.smithi050.stdout: osd_max_omap_entries_per_request = 10000 2020-05-24T13:49:17.356 INFO:tasks.ceph:Creating admin key on c2.mon.a... 2020-05-24T13:49:17.356 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c2.keyring 2020-05-24T13:49:17.516 INFO:tasks.ceph:Copying monmap to all nodes... 2020-05-24T13:49:17.600 DEBUG:teuthology.orchestra.remote:smithi160:/etc/ceph/c2.keyring is 206B 2020-05-24T13:49:17.729 DEBUG:teuthology.orchestra.remote:smithi160:/home/ubuntu/cephtest/c2.monmap is 177B 2020-05-24T13:49:17.788 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi050.front.sepia.ceph.com 2020-05-24T13:49:17.789 INFO:teuthology.orchestra.run.smithi050:> sudo sh -c 'cat > /etc/ceph/c2.keyring' && sudo chmod 0644 /etc/ceph/c2.keyring 2020-05-24T13:49:17.937 INFO:teuthology.orchestra.run.smithi050:> cat > /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:18.008 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:49:18.008 INFO:teuthology.orchestra.run.smithi160:> sudo sh -c 'cat > /etc/ceph/c2.keyring' && sudo chmod 0644 /etc/ceph/c2.keyring 2020-05-24T13:49:18.159 INFO:teuthology.orchestra.run.smithi160:> cat > /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:18.237 INFO:tasks.ceph:Setting up mon nodes... 2020-05-24T13:49:18.238 INFO:tasks.ceph:Setting up mgr nodes... 2020-05-24T13:49:18.238 INFO:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mgr/c2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c2-x/keyring 2020-05-24T13:49:18.476 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mgr/c2-x/keyring 2020-05-24T13:49:18.481 INFO:tasks.ceph:Setting up mds nodes... 2020-05-24T13:49:18.481 INFO:tasks.ceph_client:Setting up client nodes... 2020-05-24T13:49:18.482 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c2.client.0.keyring && sudo chmod 0644 /etc/ceph/c2.client.0.keyring 2020-05-24T13:49:18.618 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/c2.client.0.keyring 2020-05-24T13:49:18.649 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c2.client.1.keyring && sudo chmod 0644 /etc/ceph/c2.client.1.keyring 2020-05-24T13:49:18.786 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/c2.client.1.keyring 2020-05-24T13:49:18.815 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-05-24T13:49:18.816 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}} 2020-05-24T13:49:18.816 INFO:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/c2-0 2020-05-24T13:49:18.922 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:49:18.923 INFO:tasks.ceph:role: c2.osd.0 2020-05-24T13:49:18.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:49:18.923 INFO:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-05-24T13:49:19.510 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:49:19.510 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:49:19.510 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:49:19.510 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:49:19.511 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:49:19.511 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:49:19.511 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:49:19.511 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:49:19.511 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:49:19.513 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2020-05-24T13:49:19.514 INFO:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c2-0 2020-05-24T13:49:19.705 INFO:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-0 2020-05-24T13:49:19.829 INFO:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/c2-1 2020-05-24T13:49:19.935 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:49:19.936 INFO:tasks.ceph:role: c2.osd.1 2020-05-24T13:49:19.936 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:49:19.936 INFO:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-05-24T13:49:20.494 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:49:20.495 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:49:20.495 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:49:20.495 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:49:20.495 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:49:20.496 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:49:20.496 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:49:20.496 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:49:20.496 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:49:20.497 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2020-05-24T13:49:20.498 INFO:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c2-1 2020-05-24T13:49:20.614 INFO:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-1 2020-05-24T13:49:20.695 INFO:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/c2-2 2020-05-24T13:49:20.761 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2020-05-24T13:49:20.761 INFO:tasks.ceph:role: c2.osd.2 2020-05-24T13:49:20.769 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com 2020-05-24T13:49:20.770 INFO:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-05-24T13:49:21.336 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-05-24T13:49:21.337 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2020-05-24T13:49:21.337 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=0, sparse=0 2020-05-24T13:49:21.338 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-05-24T13:49:21.338 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2020-05-24T13:49:21.338 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-05-24T13:49:21.339 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-05-24T13:49:21.339 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-05-24T13:49:21.339 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-05-24T13:49:21.341 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2020-05-24T13:49:21.341 INFO:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c2-2 2020-05-24T13:49:21.484 INFO:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-2 2020-05-24T13:49:21.604 INFO:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:21.787 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:21.787 7feb8794d9c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-0/keyring: can't open /var/lib/ceph/osd/c2-0/keyring: (2) No such file or directory 2020-05-24T13:49:21.789 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:21.789 7feb8794d9c0 -1 created new key in keyring /var/lib/ceph/osd/c2-0/keyring 2020-05-24T13:49:21.790 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:21.789 7feb8794d9c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2020-05-24T13:49:21.790 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:21.789 7feb8794d9c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2020-05-24T13:49:21.790 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:21.789 7feb8794d9c0 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2020-05-24T13:49:21.797 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:21.797 7feb8794d9c0 -1 bluestore(/var/lib/ceph/osd/c2-0) _read_fsid unparsable uuid 2020-05-24T13:49:23.847 INFO:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-0 2020-05-24T13:49:23.970 INFO:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:24.156 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:24.156 7f21db7359c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-1/keyring: can't open /var/lib/ceph/osd/c2-1/keyring: (2) No such file or directory 2020-05-24T13:49:24.157 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:24.156 7f21db7359c0 -1 created new key in keyring /var/lib/ceph/osd/c2-1/keyring 2020-05-24T13:49:24.158 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:24.156 7f21db7359c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2020-05-24T13:49:24.158 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:24.156 7f21db7359c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2020-05-24T13:49:24.158 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:24.156 7f21db7359c0 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2020-05-24T13:49:24.159 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:24.158 7f21db7359c0 -1 bluestore(/var/lib/ceph/osd/c2-1) _read_fsid unparsable uuid 2020-05-24T13:49:26.207 INFO:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-1 2020-05-24T13:49:26.323 INFO:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:26.508 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:26.507 7f99cdac19c0 -1 auth: error reading file: /var/lib/ceph/osd/c2-2/keyring: can't open /var/lib/ceph/osd/c2-2/keyring: (2) No such file or directory 2020-05-24T13:49:26.509 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:26.508 7f99cdac19c0 -1 created new key in keyring /var/lib/ceph/osd/c2-2/keyring 2020-05-24T13:49:26.509 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:26.508 7f99cdac19c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2020-05-24T13:49:26.509 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:26.508 7f99cdac19c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2020-05-24T13:49:26.510 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:26.508 7f99cdac19c0 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2020-05-24T13:49:26.510 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:49:26.509 7f99cdac19c0 -1 bluestore(/var/lib/ceph/osd/c2-2) _read_fsid unparsable uuid 2020-05-24T13:49:28.566 INFO:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-2 2020-05-24T13:49:28.684 INFO:tasks.ceph:Reading keys from all nodes... 2020-05-24T13:49:28.685 INFO:teuthology.orchestra.run.smithi160:> mktemp 2020-05-24T13:49:28.759 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.LfKlSX7ASU 2020-05-24T13:49:28.760 INFO:teuthology.orchestra.run.smithi160:> sudo cp /var/lib/ceph/mgr/c2-x/keyring /tmp/tmp.LfKlSX7ASU 2020-05-24T13:49:28.847 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.LfKlSX7ASU 2020-05-24T13:49:29.008 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.LfKlSX7ASU is 56B 2020-05-24T13:49:29.071 INFO:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.LfKlSX7ASU 2020-05-24T13:49:29.125 INFO:teuthology.orchestra.run.smithi160:> mktemp 2020-05-24T13:49:29.182 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.ZbDPzcAfzS 2020-05-24T13:49:29.183 INFO:teuthology.orchestra.run.smithi160:> sudo cp /var/lib/ceph/osd/c2-0/keyring /tmp/tmp.ZbDPzcAfzS 2020-05-24T13:49:29.252 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.ZbDPzcAfzS 2020-05-24T13:49:29.364 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.ZbDPzcAfzS is 56B 2020-05-24T13:49:29.415 INFO:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.ZbDPzcAfzS 2020-05-24T13:49:29.462 INFO:teuthology.orchestra.run.smithi160:> mktemp 2020-05-24T13:49:29.573 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.zBnytCo936 2020-05-24T13:49:29.573 INFO:teuthology.orchestra.run.smithi160:> sudo cp /var/lib/ceph/osd/c2-1/keyring /tmp/tmp.zBnytCo936 2020-05-24T13:49:29.660 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.zBnytCo936 2020-05-24T13:49:29.799 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.zBnytCo936 is 56B 2020-05-24T13:49:29.869 INFO:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.zBnytCo936 2020-05-24T13:49:29.923 INFO:teuthology.orchestra.run.smithi160:> mktemp 2020-05-24T13:49:29.973 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.czYl1MSL45 2020-05-24T13:49:29.974 INFO:teuthology.orchestra.run.smithi160:> sudo cp /var/lib/ceph/osd/c2-2/keyring /tmp/tmp.czYl1MSL45 2020-05-24T13:49:30.051 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.czYl1MSL45 2020-05-24T13:49:30.175 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.czYl1MSL45 is 56B 2020-05-24T13:49:30.244 INFO:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.czYl1MSL45 2020-05-24T13:49:30.358 DEBUG:teuthology.orchestra.remote:smithi160:/etc/ceph/c2.client.0.keyring is 59B 2020-05-24T13:49:30.481 DEBUG:teuthology.orchestra.remote:smithi160:/etc/ceph/c2.client.1.keyring is 59B 2020-05-24T13:49:30.549 INFO:tasks.ceph:Adding keys to all mons... 2020-05-24T13:49:30.550 INFO:teuthology.orchestra.run.smithi160:> sudo tee -a /etc/ceph/c2.keyring 2020-05-24T13:49:30.620 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr.x] 2020-05-24T13:49:30.620 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBee8peNQp0HBAAYfK2PvaOVzMHbPUd0XzIIA== 2020-05-24T13:49:30.621 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.0] 2020-05-24T13:49:30.621 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBhe8pe3jT/LhAAyICddx/Jr2zXsMHGdMQZug== 2020-05-24T13:49:30.621 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.1] 2020-05-24T13:49:30.621 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBke8pe1hNjCRAAW2hzAsxmw7CF0N+67Kw5Lw== 2020-05-24T13:49:30.621 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.2] 2020-05-24T13:49:30.622 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBme8peNJtWHhAAEenZsc5LJwm60MZA6VsMPQ== 2020-05-24T13:49:30.622 INFO:teuthology.orchestra.run.smithi160.stdout:[client.0] 2020-05-24T13:49:30.622 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBee8peIaX0JBAAswahs11AdOje7Adjuu62mQ== 2020-05-24T13:49:30.622 INFO:teuthology.orchestra.run.smithi160.stdout:[client.1] 2020-05-24T13:49:30.622 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBee8pefgfuLhAAUWAJM6acG4PCouZ+81xCrA== 2020-05-24T13:49:30.623 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-05-24T13:49:30.859 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:49:31.059 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:49:31.257 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-05-24T13:49:31.457 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-24T13:49:31.636 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-05-24T13:49:31.834 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-05-24T13:49:31.834 INFO:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/c2-a 2020-05-24T13:49:31.947 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c2 --mkfs -i a --monmap /home/ubuntu/cephtest/c2.monmap --keyring /etc/ceph/c2.keyring 2020-05-24T13:49:32.106 INFO:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c2-a 2020-05-24T13:49:32.192 INFO:teuthology.orchestra.run.smithi160:> rm -- /home/ubuntu/cephtest/c2.monmap 2020-05-24T13:49:32.252 INFO:tasks.ceph:Starting mon daemons in cluster c2... 2020-05-24T13:49:32.252 DEBUG:teuthology.misc:running c2.mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2020-05-24T13:49:32.253 INFO:tasks.ceph.c2.mon.a:Restarting daemon 2020-05-24T13:49:32.253 INFO:teuthology.orchestra.run.smithi160:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.mon.a.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster c2 -i a 2020-05-24T13:49:32.258 INFO:tasks.ceph.c2.mon.a:Started 2020-05-24T13:49:32.258 INFO:tasks.ceph:Starting mgr daemons in cluster c2... 2020-05-24T13:49:32.259 INFO:tasks.ceph.c2.mgr.x:Restarting daemon 2020-05-24T13:49:32.259 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster c2 -i x 2020-05-24T13:49:32.262 INFO:tasks.ceph.c2.mgr.x:Started 2020-05-24T13:49:32.262 INFO:tasks.ceph:Setting crush tunables to default 2020-05-24T13:49:32.263 INFO:teuthology.orchestra.run.smithi160:> sudo ceph --cluster c2 osd crush tunables default 2020-05-24T13:49:37.875 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:00:05.116 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T13:49:43.879 INFO:teuthology.orchestra.run.smithi160.stderr:adjusted tunables profile to default 2020-05-24T13:49:43.893 INFO:tasks.ceph:Starting osd daemons in cluster c2... 2020-05-24T13:49:43.893 INFO:teuthology.orchestra.run.smithi160:> mktemp 2020-05-24T13:49:43.941 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.RhtrYTzBrV 2020-05-24T13:49:43.941 INFO:teuthology.orchestra.run.smithi160:> sudo cp /var/lib/ceph/osd/c2-0/fsid /tmp/tmp.RhtrYTzBrV 2020-05-24T13:49:44.005 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.RhtrYTzBrV 2020-05-24T13:49:44.107 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.RhtrYTzBrV is 37B 2020-05-24T13:49:44.154 INFO:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.RhtrYTzBrV 2020-05-24T13:49:44.198 INFO:teuthology.orchestra.run.smithi160:> mktemp 2020-05-24T13:49:44.293 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.iAyxr3faTz 2020-05-24T13:49:44.293 INFO:teuthology.orchestra.run.smithi160:> sudo cp /var/lib/ceph/osd/c2-1/fsid /tmp/tmp.iAyxr3faTz 2020-05-24T13:49:44.367 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.iAyxr3faTz 2020-05-24T13:49:44.479 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.iAyxr3faTz is 37B 2020-05-24T13:49:44.528 INFO:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.iAyxr3faTz 2020-05-24T13:49:44.570 INFO:teuthology.orchestra.run.smithi160:> mktemp 2020-05-24T13:49:44.671 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.lJeNtuyDwt 2020-05-24T13:49:44.672 INFO:teuthology.orchestra.run.smithi160:> sudo cp /var/lib/ceph/osd/c2-2/fsid /tmp/tmp.lJeNtuyDwt 2020-05-24T13:49:44.738 INFO:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.lJeNtuyDwt 2020-05-24T13:49:44.843 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.lJeNtuyDwt is 37B 2020-05-24T13:49:44.891 INFO:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.lJeNtuyDwt 2020-05-24T13:49:44.943 INFO:teuthology.orchestra.run.smithi160:> sudo ceph --cluster c2 osd new ec4f79bb-cf1b-413d-b846-f155eda204ac 0 2020-05-24T13:49:46.067 INFO:teuthology.orchestra.run.smithi160.stdout:0 2020-05-24T13:49:46.088 INFO:teuthology.orchestra.run.smithi160:> sudo ceph --cluster c2 osd new dfebc3fe-b159-4617-8f1e-f6a17766557d 1 2020-05-24T13:49:46.986 INFO:teuthology.orchestra.run.smithi160.stdout:1 2020-05-24T13:49:47.009 INFO:teuthology.orchestra.run.smithi160:> sudo ceph --cluster c2 osd new 6e1fd517-22fb-4294-94a3-60fa65db8330 2 2020-05-24T13:49:48.061 INFO:teuthology.orchestra.run.smithi160.stdout:2 2020-05-24T13:49:48.080 DEBUG:teuthology.misc:running c2.osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:49:48.081 INFO:tasks.ceph.c2.osd.0:Restarting daemon 2020-05-24T13:49:48.081 INFO:teuthology.orchestra.run.smithi160:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.osd.0.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c2 -i 0 2020-05-24T13:49:48.084 INFO:tasks.ceph.c2.osd.0:Started 2020-05-24T13:49:48.085 DEBUG:teuthology.misc:running c2.osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:49:48.085 INFO:tasks.ceph.c2.osd.1:Restarting daemon 2020-05-24T13:49:48.086 INFO:teuthology.orchestra.run.smithi160:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.osd.1.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c2 -i 1 2020-05-24T13:49:48.089 INFO:tasks.ceph.c2.osd.1:Started 2020-05-24T13:49:48.089 DEBUG:teuthology.misc:running c2.osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2020-05-24T13:49:48.090 INFO:tasks.ceph.c2.osd.2:Restarting daemon 2020-05-24T13:49:48.090 INFO:teuthology.orchestra.run.smithi160:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.osd.2.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-osd -f --cluster c2 -i 2 2020-05-24T13:49:48.093 INFO:tasks.ceph.c2.osd.2:Started 2020-05-24T13:49:48.094 INFO:tasks.ceph:Waiting for OSDs to come up 2020-05-24T13:49:48.398 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2020-05-24T13:49:49.562 INFO:teuthology.misc.health.smithi160.stdout: 2020-05-24T13:49:49.562 INFO:teuthology.misc.health.smithi160.stdout:{"epoch":5,"fsid":"e6d3bc87-1c02-4a97-acf9-0c849318394c","created":"2020-05-24 13:49:40.402235","modified":"2020-05-24 13:49:47.961839","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"ec4f79bb-cf1b-413d-b846-f155eda204ac","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"dfebc3fe-b159-4617-8f1e-f6a17766557d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"6e1fd517-22fb-4294-94a3-60fa65db8330","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:49.579 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-05-24T13:49:54.594 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:==00:00:00:06.360 10627== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:54.621 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:==00:00:00:06.400 10625== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:54.649 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:==00:00:00:06.414 10627== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:54.657 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:==00:00:00:06.425 10626== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:54.675 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:==00:00:00:06.454 10625== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:54.717 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:==00:00:00:06.484 10626== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:55.273 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:2020-05-24 13:49:55.239 9ae5e40 -1 Falling back to public interface 2020-05-24T13:49:55.349 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:2020-05-24 13:49:55.317 9ae5e40 -1 Falling back to public interface 2020-05-24T13:49:55.362 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:2020-05-24 13:49:55.339 9ae5e40 -1 Falling back to public interface 2020-05-24T13:49:55.483 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:==00:00:00:07.249 10627== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:55.689 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:==00:00:00:07.468 10625== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:55.718 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:==00:00:00:07.483 10626== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:55.882 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2020-05-24T13:49:55.897 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:==00:00:00:07.663 10627== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:55.916 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:==00:00:00:07.682 10627== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:56.062 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:==00:00:00:07.830 10626== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:56.075 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:==00:00:00:07.843 10626== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:56.078 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:==00:00:00:07.857 10625== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:56.088 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:==00:00:00:07.867 10625== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:56.632 INFO:teuthology.misc.health.smithi160.stdout: 2020-05-24T13:49:56.633 INFO:teuthology.misc.health.smithi160.stdout:{"epoch":5,"fsid":"e6d3bc87-1c02-4a97-acf9-0c849318394c","created":"2020-05-24 13:49:40.402235","modified":"2020-05-24 13:49:47.961839","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"ec4f79bb-cf1b-413d-b846-f155eda204ac","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":1,"uuid":"dfebc3fe-b159-4617-8f1e-f6a17766557d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]},{"osd":2,"uuid":"6e1fd517-22fb-4294-94a3-60fa65db8330","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":":/0","cluster_addr":":/0","heartbeat_back_addr":":/0","heartbeat_front_addr":":/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:49:56.643 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-05-24T13:49:56.897 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:==00:00:00:08.665 10626== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:57.012 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:==00:00:00:08.777 10627== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:57.139 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:==00:00:00:08.918 10625== Warning: unimplemented fcntl command: 1038 2020-05-24T13:49:58.325 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:2020-05-24 13:49:58.321 9ae5e40 -1 osd.2 0 log_to_monitors {default=true} 2020-05-24T13:49:58.382 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:2020-05-24 13:49:58.378 9ae5e40 -1 osd.1 0 log_to_monitors {default=true} 2020-05-24T13:49:58.783 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:2020-05-24 13:49:58.780 9ae5e40 -1 osd.0 0 log_to_monitors {default=true} 2020-05-24T13:50:00.102 INFO:tasks.ceph.c2.osd.2.smithi160.stderr:2020-05-24 13:50:00.097 28cbc700 -1 osd.2 0 waiting for initial osdmap 2020-05-24T13:50:00.202 INFO:tasks.ceph.c2.osd.1.smithi160.stderr:2020-05-24 13:50:00.188 28cbc700 -1 osd.1 0 waiting for initial osdmap 2020-05-24T13:50:01.270 INFO:tasks.ceph.c2.osd.0.smithi160.stderr:2020-05-24 13:50:01.264 28cbc700 -1 osd.0 0 waiting for initial osdmap 2020-05-24T13:50:02.946 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:03.727 INFO:teuthology.misc.health.smithi160.stdout: 2020-05-24T13:50:03.728 INFO:teuthology.misc.health.smithi160.stdout:{"epoch":9,"fsid":"e6d3bc87-1c02-4a97-acf9-0c849318394c","created":"2020-05-24 13:49:40.402235","modified":"2020-05-24 13:50:02.342612","last_up_change":"2020-05-24 13:50:02.342612","last_in_change":"2020-05-24 13:50:02.342612","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[],"osds":[{"osd":0,"uuid":"ec4f79bb-cf1b-413d-b846-f155eda204ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":10625},{"type":"v1","addr":"172.21.15.160:6811","nonce":10625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6813","nonce":10625},{"type":"v1","addr":"172.21.15.160:6814","nonce":10625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":10625},{"type":"v1","addr":"172.21.15.160:6821","nonce":10625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":10625},{"type":"v1","addr":"172.21.15.160:6818","nonce":10625}]},"public_addr":"172.21.15.160:6811/10625","cluster_addr":"172.21.15.160:6814/10625","heartbeat_back_addr":"172.21.15.160:6821/10625","heartbeat_front_addr":"172.21.15.160:6818/10625","state":["exists","up"]},{"osd":1,"uuid":"dfebc3fe-b159-4617-8f1e-f6a17766557d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":10627},{"type":"v1","addr":"172.21.15.160:6803","nonce":10627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":10627},{"type":"v1","addr":"172.21.15.160:6805","nonce":10627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":10627},{"type":"v1","addr":"172.21.15.160:6809","nonce":10627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":10627},{"type":"v1","addr":"172.21.15.160:6807","nonce":10627}]},"public_addr":"172.21.15.160:6803/10627","cluster_addr":"172.21.15.160:6805/10627","heartbeat_back_addr":"172.21.15.160:6809/10627","heartbeat_front_addr":"172.21.15.160:6807/10627","state":["exists","up"]},{"osd":2,"uuid":"6e1fd517-22fb-4294-94a3-60fa65db8330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":10626},{"type":"v1","addr":"172.21.15.160:6815","nonce":10626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":10626},{"type":"v1","addr":"172.21.15.160:6819","nonce":10626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":10626},{"type":"v1","addr":"172.21.15.160:6825","nonce":10626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":10626},{"type":"v1","addr":"172.21.15.160:6823","nonce":10626}]},"public_addr":"172.21.15.160:6815/10626","cluster_addr":"172.21.15.160:6819/10626","heartbeat_back_addr":"172.21.15.160:6825/10626","heartbeat_front_addr":"172.21.15.160:6823/10626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:03.746 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-05-24T13:50:03.747 INFO:tasks.ceph:Creating RBD pool 2020-05-24T13:50:03.747 INFO:teuthology.orchestra.run.smithi160:> sudo ceph --cluster c2 osd pool create rbd 8 2020-05-24T13:50:05.905 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'rbd' created 2020-05-24T13:50:05.932 INFO:teuthology.orchestra.run.smithi160:> sudo ceph --cluster c2 osd pool application enable rbd rbd --yes-i-really-mean-it 2020-05-24T13:50:07.893 INFO:teuthology.orchestra.run.smithi160.stderr:enabled application 'rbd' on pool 'rbd' 2020-05-24T13:50:07.913 INFO:tasks.ceph:Starting mds daemons in cluster c2... 2020-05-24T13:50:07.914 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:08.735 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:50:08.736 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":12,"fsid":"e6d3bc87-1c02-4a97-acf9-0c849318394c","created":"2020-05-24 13:49:40.402235","modified":"2020-05-24 13:50:07.784471","last_up_change":"2020-05-24 13:50:02.342612","last_in_change":"2020-05-24 13:50:02.342612","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:50:04.680522","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ec4f79bb-cf1b-413d-b846-f155eda204ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":10625},{"type":"v1","addr":"172.21.15.160:6811","nonce":10625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6813","nonce":10625},{"type":"v1","addr":"172.21.15.160:6814","nonce":10625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":10625},{"type":"v1","addr":"172.21.15.160:6821","nonce":10625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":10625},{"type":"v1","addr":"172.21.15.160:6818","nonce":10625}]},"public_addr":"172.21.15.160:6811/10625","cluster_addr":"172.21.15.160:6814/10625","heartbeat_back_addr":"172.21.15.160:6821/10625","heartbeat_front_addr":"172.21.15.160:6818/10625","state":["exists","up"]},{"osd":1,"uuid":"dfebc3fe-b159-4617-8f1e-f6a17766557d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":10627},{"type":"v1","addr":"172.21.15.160:6803","nonce":10627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":10627},{"type":"v1","addr":"172.21.15.160:6805","nonce":10627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":10627},{"type":"v1","addr":"172.21.15.160:6809","nonce":10627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":10627},{"type":"v1","addr":"172.21.15.160:6807","nonce":10627}]},"public_addr":"172.21.15.160:6803/10627","cluster_addr":"172.21.15.160:6805/10627","heartbeat_back_addr":"172.21.15.160:6809/10627","heartbeat_front_addr":"172.21.15.160:6807/10627","state":["exists","up"]},{"osd":2,"uuid":"6e1fd517-22fb-4294-94a3-60fa65db8330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":10626},{"type":"v1","addr":"172.21.15.160:6815","nonce":10626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":10626},{"type":"v1","addr":"172.21.15.160:6819","nonce":10626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":10626},{"type":"v1","addr":"172.21.15.160:6825","nonce":10626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":10626},{"type":"v1","addr":"172.21.15.160:6823","nonce":10626}]},"public_addr":"172.21.15.160:6815/10626","cluster_addr":"172.21.15.160:6819/10626","heartbeat_back_addr":"172.21.15.160:6825/10626","heartbeat_front_addr":"172.21.15.160:6823/10626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:08.761 INFO:tasks.ceph.ceph_manager.c2:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2020-05-24 13:50:04.680522', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'warn', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-05-24T13:50:08.762 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd pool get rbd pg_num 2020-05-24T13:50:09.659 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 8 2020-05-24T13:50:09.683 INFO:tasks.ceph:Waiting until c2 daemons up and pgs clean... 2020-05-24T13:50:09.684 INFO:tasks.ceph.ceph_manager.c2:waiting for mgr available 2020-05-24T13:50:09.684 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 mgr dump --format=json 2020-05-24T13:50:10.710 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:50:10.736 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":3,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":9734},{"type":"v1","addr":"172.21.15.160:6801","nonce":9734}]},"active_addr":"172.21.15.160:6801/9734","active_change":"2020-05-24 13:49:45.331777","available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_url":{"name":"server_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_iterations":{"name":"upmap_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimization iterations","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout_min":{"name":"inventory_cache_timeout_min","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"]}} 2020-05-24T13:50:10.738 INFO:tasks.ceph.ceph_manager.c2:mgr available! 2020-05-24T13:50:11.041 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:11.804 INFO:teuthology.misc.health.smithi160.stdout: 2020-05-24T13:50:11.805 INFO:teuthology.misc.health.smithi160.stdout:{"epoch":13,"fsid":"e6d3bc87-1c02-4a97-acf9-0c849318394c","created":"2020-05-24 13:49:40.402235","modified":"2020-05-24 13:50:08.975367","last_up_change":"2020-05-24 13:50:02.342612","last_in_change":"2020-05-24 13:50:02.342612","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:50:04.680522","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ec4f79bb-cf1b-413d-b846-f155eda204ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":10625},{"type":"v1","addr":"172.21.15.160:6811","nonce":10625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6813","nonce":10625},{"type":"v1","addr":"172.21.15.160:6814","nonce":10625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":10625},{"type":"v1","addr":"172.21.15.160:6821","nonce":10625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":10625},{"type":"v1","addr":"172.21.15.160:6818","nonce":10625}]},"public_addr":"172.21.15.160:6811/10625","cluster_addr":"172.21.15.160:6814/10625","heartbeat_back_addr":"172.21.15.160:6821/10625","heartbeat_front_addr":"172.21.15.160:6818/10625","state":["exists","up"]},{"osd":1,"uuid":"dfebc3fe-b159-4617-8f1e-f6a17766557d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":10627},{"type":"v1","addr":"172.21.15.160:6803","nonce":10627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":10627},{"type":"v1","addr":"172.21.15.160:6805","nonce":10627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":10627},{"type":"v1","addr":"172.21.15.160:6809","nonce":10627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":10627},{"type":"v1","addr":"172.21.15.160:6807","nonce":10627}]},"public_addr":"172.21.15.160:6803/10627","cluster_addr":"172.21.15.160:6805/10627","heartbeat_back_addr":"172.21.15.160:6809/10627","heartbeat_front_addr":"172.21.15.160:6807/10627","state":["exists","up"]},{"osd":2,"uuid":"6e1fd517-22fb-4294-94a3-60fa65db8330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":10626},{"type":"v1","addr":"172.21.15.160:6815","nonce":10626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":10626},{"type":"v1","addr":"172.21.15.160:6819","nonce":10626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":10626},{"type":"v1","addr":"172.21.15.160:6825","nonce":10626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":10626},{"type":"v1","addr":"172.21.15.160:6823","nonce":10626}]},"public_addr":"172.21.15.160:6815/10626","cluster_addr":"172.21.15.160:6819/10626","heartbeat_back_addr":"172.21.15.160:6825/10626","heartbeat_front_addr":"172.21.15.160:6823/10626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:11.821 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-05-24T13:50:11.822 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd dump --format=json 2020-05-24T13:50:12.891 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:50:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":13,"fsid":"e6d3bc87-1c02-4a97-acf9-0c849318394c","created":"2020-05-24 13:49:40.402235","modified":"2020-05-24 13:50:08.975367","last_up_change":"2020-05-24 13:50:02.342612","last_in_change":"2020-05-24 13:50:02.342612","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"nautilus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-05-24 13:50:04.680522","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"warn","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ec4f79bb-cf1b-413d-b846-f155eda204ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":10625},{"type":"v1","addr":"172.21.15.160:6811","nonce":10625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6813","nonce":10625},{"type":"v1","addr":"172.21.15.160:6814","nonce":10625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":10625},{"type":"v1","addr":"172.21.15.160:6821","nonce":10625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":10625},{"type":"v1","addr":"172.21.15.160:6818","nonce":10625}]},"public_addr":"172.21.15.160:6811/10625","cluster_addr":"172.21.15.160:6814/10625","heartbeat_back_addr":"172.21.15.160:6821/10625","heartbeat_front_addr":"172.21.15.160:6818/10625","state":["exists","up"]},{"osd":1,"uuid":"dfebc3fe-b159-4617-8f1e-f6a17766557d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":10627},{"type":"v1","addr":"172.21.15.160:6803","nonce":10627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":10627},{"type":"v1","addr":"172.21.15.160:6805","nonce":10627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":10627},{"type":"v1","addr":"172.21.15.160:6809","nonce":10627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":10627},{"type":"v1","addr":"172.21.15.160:6807","nonce":10627}]},"public_addr":"172.21.15.160:6803/10627","cluster_addr":"172.21.15.160:6805/10627","heartbeat_back_addr":"172.21.15.160:6809/10627","heartbeat_front_addr":"172.21.15.160:6807/10627","state":["exists","up"]},{"osd":2,"uuid":"6e1fd517-22fb-4294-94a3-60fa65db8330","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":10626},{"type":"v1","addr":"172.21.15.160:6815","nonce":10626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":10626},{"type":"v1","addr":"172.21.15.160:6819","nonce":10626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":10626},{"type":"v1","addr":"172.21.15.160:6825","nonce":10626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":10626},{"type":"v1","addr":"172.21.15.160:6823","nonce":10626}]},"public_addr":"172.21.15.160:6815/10626","cluster_addr":"172.21.15.160:6819/10626","heartbeat_back_addr":"172.21.15.160:6825/10626","heartbeat_front_addr":"172.21.15.160:6823/10626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4611087854031667199,"old_weight":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-05-24T13:50:12.916 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 tell osd.0 flush_pg_stats 2020-05-24T13:50:13.663 INFO:teuthology.orchestra.run.smithi160.stdout:38654705670 2020-05-24T13:50:13.676 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 tell osd.1 flush_pg_stats 2020-05-24T13:50:14.214 INFO:teuthology.orchestra.run.smithi160.stdout:34359738372 2020-05-24T13:50:14.230 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 tell osd.2 flush_pg_stats 2020-05-24T13:50:14.837 INFO:teuthology.orchestra.run.smithi160.stdout:34359738373 2020-05-24T13:50:14.860 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd last-stat-seq osd.0 2020-05-24T13:50:15.809 INFO:teuthology.orchestra.run.smithi160.stdout:38654705670 2020-05-24T13:50:15.824 INFO:tasks.ceph.ceph_manager.c2:need seq 38654705670 got 38654705670 for osd.0 2020-05-24T13:50:15.824 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd last-stat-seq osd.1 2020-05-24T13:50:16.635 INFO:teuthology.orchestra.run.smithi160.stdout:34359738372 2020-05-24T13:50:16.655 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738372 got 34359738372 for osd.1 2020-05-24T13:50:16.656 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 osd last-stat-seq osd.2 2020-05-24T13:50:17.674 INFO:teuthology.orchestra.run.smithi160.stdout:34359738373 2020-05-24T13:50:17.686 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738373 for osd.2 2020-05-24T13:50:17.687 INFO:tasks.ceph.ceph_manager.c2:waiting for clean 2020-05-24T13:50:17.688 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 pg dump --format=json 2020-05-24T13:50:18.726 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:50:18.726 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2020-05-24T13:50:18.738 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-05-24 13:50:17.725591","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"kb":314572800,"kb_used":3146088,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426712,"statfs":{"total":322122547200,"available":318900953088,"internally_reserved":3221225472,"allocated":344064,"data_stored":87765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.503198","last_change":"2020-05-24 13:50:08.385030","last_active":"2020-05-24 13:50:08.503198","last_peered":"2020-05-24 13:50:08.503198","last_clean":"2020-05-24 13:50:08.503198","last_became_active":"2020-05-24 13:50:08.361926","last_became_peered":"2020-05-24 13:50:08.361926","last_unstale":"2020-05-24 13:50:08.503198","last_undegraded":"2020-05-24 13:50:08.503198","last_fullsized":"2020-05-24 13:50:08.503198","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.504989","last_change":"2020-05-24 13:50:08.475319","last_active":"2020-05-24 13:50:08.504989","last_peered":"2020-05-24 13:50:08.504989","last_clean":"2020-05-24 13:50:08.504989","last_became_active":"2020-05-24 13:50:08.461478","last_became_peered":"2020-05-24 13:50:08.461478","last_unstale":"2020-05-24 13:50:08.504989","last_undegraded":"2020-05-24 13:50:08.504989","last_fullsized":"2020-05-24 13:50:08.504989","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.397282","last_change":"2020-05-24 13:50:08.320760","last_active":"2020-05-24 13:50:08.397282","last_peered":"2020-05-24 13:50:08.397282","last_clean":"2020-05-24 13:50:08.397282","last_became_active":"2020-05-24 13:50:08.269914","last_became_peered":"2020-05-24 13:50:08.269914","last_unstale":"2020-05-24 13:50:08.397282","last_undegraded":"2020-05-24 13:50:08.397282","last_fullsized":"2020-05-24 13:50:08.397282","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.494923","last_change":"2020-05-24 13:50:08.334679","last_active":"2020-05-24 13:50:08.494923","last_peered":"2020-05-24 13:50:08.494923","last_clean":"2020-05-24 13:50:08.494923","last_became_active":"2020-05-24 13:50:08.310979","last_became_peered":"2020-05-24 13:50:08.310979","last_unstale":"2020-05-24 13:50:08.494923","last_undegraded":"2020-05-24 13:50:08.494923","last_fullsized":"2020-05-24 13:50:08.494923","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.504325","last_change":"2020-05-24 13:50:08.434615","last_active":"2020-05-24 13:50:08.504325","last_peered":"2020-05-24 13:50:08.504325","last_clean":"2020-05-24 13:50:08.504325","last_became_active":"2020-05-24 13:50:08.420917","last_became_peered":"2020-05-24 13:50:08.420917","last_unstale":"2020-05-24 13:50:08.504325","last_undegraded":"2020-05-24 13:50:08.504325","last_fullsized":"2020-05-24 13:50:08.504325","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.424303","last_change":"2020-05-24 13:50:08.334106","last_active":"2020-05-24 13:50:08.424303","last_peered":"2020-05-24 13:50:08.424303","last_clean":"2020-05-24 13:50:08.424303","last_became_active":"2020-05-24 13:50:08.281558","last_became_peered":"2020-05-24 13:50:08.281558","last_unstale":"2020-05-24 13:50:08.424303","last_undegraded":"2020-05-24 13:50:08.424303","last_fullsized":"2020-05-24 13:50:08.424303","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.118771","last_change":"2020-05-24 13:50:07.364556","last_active":"2020-05-24 13:50:08.118771","last_peered":"2020-05-24 13:50:08.118771","last_clean":"2020-05-24 13:50:08.118771","last_became_active":"2020-05-24 13:50:07.331102","last_became_peered":"2020-05-24 13:50:07.331102","last_unstale":"2020-05-24 13:50:08.118771","last_undegraded":"2020-05-24 13:50:08.118771","last_fullsized":"2020-05-24 13:50:08.118771","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.496644","last_change":"2020-05-24 13:50:08.352852","last_active":"2020-05-24 13:50:08.496644","last_peered":"2020-05-24 13:50:08.496644","last_clean":"2020-05-24 13:50:08.496644","last_became_active":"2020-05-24 13:50:08.272282","last_became_peered":"2020-05-24 13:50:08.272282","last_unstale":"2020-05-24 13:50:08.496644","last_undegraded":"2020-05-24 13:50:08.496644","last_fullsized":"2020-05-24 13:50:08.496644","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}]}} 2020-05-24T13:50:18.739 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 pg dump --format=json 2020-05-24T13:50:19.628 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2020-05-24T13:50:19.629 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:50:19.646 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-05-24 13:50:17.725591","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"kb":314572800,"kb_used":3146088,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426712,"statfs":{"total":322122547200,"available":318900953088,"internally_reserved":3221225472,"allocated":344064,"data_stored":87765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.503198","last_change":"2020-05-24 13:50:08.385030","last_active":"2020-05-24 13:50:08.503198","last_peered":"2020-05-24 13:50:08.503198","last_clean":"2020-05-24 13:50:08.503198","last_became_active":"2020-05-24 13:50:08.361926","last_became_peered":"2020-05-24 13:50:08.361926","last_unstale":"2020-05-24 13:50:08.503198","last_undegraded":"2020-05-24 13:50:08.503198","last_fullsized":"2020-05-24 13:50:08.503198","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.504989","last_change":"2020-05-24 13:50:08.475319","last_active":"2020-05-24 13:50:08.504989","last_peered":"2020-05-24 13:50:08.504989","last_clean":"2020-05-24 13:50:08.504989","last_became_active":"2020-05-24 13:50:08.461478","last_became_peered":"2020-05-24 13:50:08.461478","last_unstale":"2020-05-24 13:50:08.504989","last_undegraded":"2020-05-24 13:50:08.504989","last_fullsized":"2020-05-24 13:50:08.504989","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.397282","last_change":"2020-05-24 13:50:08.320760","last_active":"2020-05-24 13:50:08.397282","last_peered":"2020-05-24 13:50:08.397282","last_clean":"2020-05-24 13:50:08.397282","last_became_active":"2020-05-24 13:50:08.269914","last_became_peered":"2020-05-24 13:50:08.269914","last_unstale":"2020-05-24 13:50:08.397282","last_undegraded":"2020-05-24 13:50:08.397282","last_fullsized":"2020-05-24 13:50:08.397282","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.494923","last_change":"2020-05-24 13:50:08.334679","last_active":"2020-05-24 13:50:08.494923","last_peered":"2020-05-24 13:50:08.494923","last_clean":"2020-05-24 13:50:08.494923","last_became_active":"2020-05-24 13:50:08.310979","last_became_peered":"2020-05-24 13:50:08.310979","last_unstale":"2020-05-24 13:50:08.494923","last_undegraded":"2020-05-24 13:50:08.494923","last_fullsized":"2020-05-24 13:50:08.494923","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.504325","last_change":"2020-05-24 13:50:08.434615","last_active":"2020-05-24 13:50:08.504325","last_peered":"2020-05-24 13:50:08.504325","last_clean":"2020-05-24 13:50:08.504325","last_became_active":"2020-05-24 13:50:08.420917","last_became_peered":"2020-05-24 13:50:08.420917","last_unstale":"2020-05-24 13:50:08.504325","last_undegraded":"2020-05-24 13:50:08.504325","last_fullsized":"2020-05-24 13:50:08.504325","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.424303","last_change":"2020-05-24 13:50:08.334106","last_active":"2020-05-24 13:50:08.424303","last_peered":"2020-05-24 13:50:08.424303","last_clean":"2020-05-24 13:50:08.424303","last_became_active":"2020-05-24 13:50:08.281558","last_became_peered":"2020-05-24 13:50:08.281558","last_unstale":"2020-05-24 13:50:08.424303","last_undegraded":"2020-05-24 13:50:08.424303","last_fullsized":"2020-05-24 13:50:08.424303","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.118771","last_change":"2020-05-24 13:50:07.364556","last_active":"2020-05-24 13:50:08.118771","last_peered":"2020-05-24 13:50:08.118771","last_clean":"2020-05-24 13:50:08.118771","last_became_active":"2020-05-24 13:50:07.331102","last_became_peered":"2020-05-24 13:50:07.331102","last_unstale":"2020-05-24 13:50:08.118771","last_undegraded":"2020-05-24 13:50:08.118771","last_fullsized":"2020-05-24 13:50:08.118771","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"12","state":"active+clean","last_fresh":"2020-05-24 13:50:08.496644","last_change":"2020-05-24 13:50:08.352852","last_active":"2020-05-24 13:50:08.496644","last_peered":"2020-05-24 13:50:08.496644","last_clean":"2020-05-24 13:50:08.496644","last_became_active":"2020-05-24 13:50:08.272282","last_became_peered":"2020-05-24 13:50:08.272282","last_unstale":"2020-05-24 13:50:08.496644","last_undegraded":"2020-05-24 13:50:08.496644","last_fullsized":"2020-05-24 13:50:08.496644","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-05-24 13:50:05.576665","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-05-24 13:50:05.576665","last_clean_scrub_stamp":"2020-05-24 13:50:05.576665","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"kb":104857600,"kb_used":1048696,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808904,"statfs":{"total":107374182400,"available":106300317696,"internally_reserved":1073741824,"allocated":114688,"data_stored":29255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}]}} 2020-05-24T13:50:19.649 INFO:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster c2 status --format=json-pretty 2020-05-24T13:50:20.691 INFO:teuthology.orchestra.run.smithi160.stdout: 2020-05-24T13:50:20.692 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2020-05-24T13:50:20.692 INFO:teuthology.orchestra.run.smithi160.stdout: "fsid": "e6d3bc87-1c02-4a97-acf9-0c849318394c", 2020-05-24T13:50:20.692 INFO:teuthology.orchestra.run.smithi160.stdout: "health": { 2020-05-24T13:50:20.693 INFO:teuthology.orchestra.run.smithi160.stdout: "checks": {}, 2020-05-24T13:50:20.693 INFO:teuthology.orchestra.run.smithi160.stdout: "status": "HEALTH_OK" 2020-05-24T13:50:20.693 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.693 INFO:teuthology.orchestra.run.smithi160.stdout: "election_epoch": 3, 2020-05-24T13:50:20.694 INFO:teuthology.orchestra.run.smithi160.stdout: "quorum": [ 2020-05-24T13:50:20.694 INFO:teuthology.orchestra.run.smithi160.stdout: 0 2020-05-24T13:50:20.694 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2020-05-24T13:50:20.694 INFO:teuthology.orchestra.run.smithi160.stdout: "quorum_names": [ 2020-05-24T13:50:20.695 INFO:teuthology.orchestra.run.smithi160.stdout: "a" 2020-05-24T13:50:20.695 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2020-05-24T13:50:20.695 INFO:teuthology.orchestra.run.smithi160.stdout: "quorum_age": 40, 2020-05-24T13:50:20.696 INFO:teuthology.orchestra.run.smithi160.stdout: "monmap": { 2020-05-24T13:50:20.696 INFO:teuthology.orchestra.run.smithi160.stdout: "epoch": 1, 2020-05-24T13:50:20.696 INFO:teuthology.orchestra.run.smithi160.stdout: "fsid": "e6d3bc87-1c02-4a97-acf9-0c849318394c", 2020-05-24T13:50:20.696 INFO:teuthology.orchestra.run.smithi160.stdout: "modified": "2020-05-24 13:49:17.140145", 2020-05-24T13:50:20.697 INFO:teuthology.orchestra.run.smithi160.stdout: "created": "2020-05-24 13:49:17.140145", 2020-05-24T13:50:20.697 INFO:teuthology.orchestra.run.smithi160.stdout: "min_mon_release": 14, 2020-05-24T13:50:20.697 INFO:teuthology.orchestra.run.smithi160.stdout: "min_mon_release_name": "nautilus", 2020-05-24T13:50:20.698 INFO:teuthology.orchestra.run.smithi160.stdout: "features": { 2020-05-24T13:50:20.698 INFO:teuthology.orchestra.run.smithi160.stdout: "persistent": [ 2020-05-24T13:50:20.698 INFO:teuthology.orchestra.run.smithi160.stdout: "kraken", 2020-05-24T13:50:20.699 INFO:teuthology.orchestra.run.smithi160.stdout: "luminous", 2020-05-24T13:50:20.699 INFO:teuthology.orchestra.run.smithi160.stdout: "mimic", 2020-05-24T13:50:20.699 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap-prune", 2020-05-24T13:50:20.699 INFO:teuthology.orchestra.run.smithi160.stdout: "nautilus" 2020-05-24T13:50:20.699 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2020-05-24T13:50:20.700 INFO:teuthology.orchestra.run.smithi160.stdout: "optional": [] 2020-05-24T13:50:20.700 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.700 INFO:teuthology.orchestra.run.smithi160.stdout: "mons": [ 2020-05-24T13:50:20.700 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.700 INFO:teuthology.orchestra.run.smithi160.stdout: "rank": 0, 2020-05-24T13:50:20.701 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "a", 2020-05-24T13:50:20.701 INFO:teuthology.orchestra.run.smithi160.stdout: "public_addrs": { 2020-05-24T13:50:20.701 INFO:teuthology.orchestra.run.smithi160.stdout: "addrvec": [ 2020-05-24T13:50:20.701 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.701 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v2", 2020-05-24T13:50:20.702 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:3300", 2020-05-24T13:50:20.702 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 0 2020-05-24T13:50:20.702 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.702 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.702 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2020-05-24T13:50:20.702 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:6789", 2020-05-24T13:50:20.703 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 0 2020-05-24T13:50:20.703 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.703 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2020-05-24T13:50:20.703 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.703 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:6789/0", 2020-05-24T13:50:20.704 INFO:teuthology.orchestra.run.smithi160.stdout: "public_addr": "172.21.15.160:6789/0" 2020-05-24T13:50:20.704 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.704 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2020-05-24T13:50:20.705 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.705 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap": { 2020-05-24T13:50:20.705 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap": { 2020-05-24T13:50:20.705 INFO:teuthology.orchestra.run.smithi160.stdout: "epoch": 13, 2020-05-24T13:50:20.705 INFO:teuthology.orchestra.run.smithi160.stdout: "num_osds": 3, 2020-05-24T13:50:20.706 INFO:teuthology.orchestra.run.smithi160.stdout: "num_up_osds": 3, 2020-05-24T13:50:20.707 INFO:teuthology.orchestra.run.smithi160.stdout: "num_in_osds": 3, 2020-05-24T13:50:20.707 INFO:teuthology.orchestra.run.smithi160.stdout: "num_remapped_pgs": 0 2020-05-24T13:50:20.707 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.707 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.708 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap": { 2020-05-24T13:50:20.708 INFO:teuthology.orchestra.run.smithi160.stdout: "pgs_by_state": [ 2020-05-24T13:50:20.708 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.709 INFO:teuthology.orchestra.run.smithi160.stdout: "state_name": "active+clean", 2020-05-24T13:50:20.709 INFO:teuthology.orchestra.run.smithi160.stdout: "count": 8 2020-05-24T13:50:20.709 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.710 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2020-05-24T13:50:20.710 INFO:teuthology.orchestra.run.smithi160.stdout: "num_pgs": 8, 2020-05-24T13:50:20.710 INFO:teuthology.orchestra.run.smithi160.stdout: "num_pools": 1, 2020-05-24T13:50:20.711 INFO:teuthology.orchestra.run.smithi160.stdout: "num_objects": 0, 2020-05-24T13:50:20.711 INFO:teuthology.orchestra.run.smithi160.stdout: "data_bytes": 0, 2020-05-24T13:50:20.711 INFO:teuthology.orchestra.run.smithi160.stdout: "bytes_used": 3221594112, 2020-05-24T13:50:20.711 INFO:teuthology.orchestra.run.smithi160.stdout: "bytes_avail": 318900953088, 2020-05-24T13:50:20.711 INFO:teuthology.orchestra.run.smithi160.stdout: "bytes_total": 322122547200 2020-05-24T13:50:20.712 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.712 INFO:teuthology.orchestra.run.smithi160.stdout: "fsmap": { 2020-05-24T13:50:20.712 INFO:teuthology.orchestra.run.smithi160.stdout: "epoch": 1, 2020-05-24T13:50:20.712 INFO:teuthology.orchestra.run.smithi160.stdout: "by_rank": [], 2020-05-24T13:50:20.712 INFO:teuthology.orchestra.run.smithi160.stdout: "up:standby": 0 2020-05-24T13:50:20.713 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.713 INFO:teuthology.orchestra.run.smithi160.stdout: "mgrmap": { 2020-05-24T13:50:20.713 INFO:teuthology.orchestra.run.smithi160.stdout: "epoch": 3, 2020-05-24T13:50:20.713 INFO:teuthology.orchestra.run.smithi160.stdout: "active_gid": 4100, 2020-05-24T13:50:20.713 INFO:teuthology.orchestra.run.smithi160.stdout: "active_name": "x", 2020-05-24T13:50:20.713 INFO:teuthology.orchestra.run.smithi160.stdout: "active_addrs": { 2020-05-24T13:50:20.714 INFO:teuthology.orchestra.run.smithi160.stdout: "addrvec": [ 2020-05-24T13:50:20.714 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.714 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v2", 2020-05-24T13:50:20.719 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:6800", 2020-05-24T13:50:20.719 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 9734 2020-05-24T13:50:20.720 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.720 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.720 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2020-05-24T13:50:20.720 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "172.21.15.160:6801", 2020-05-24T13:50:20.720 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 9734 2020-05-24T13:50:20.721 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.721 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2020-05-24T13:50:20.721 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.721 INFO:teuthology.orchestra.run.smithi160.stdout: "active_addr": "172.21.15.160:6801/9734", 2020-05-24T13:50:20.721 INFO:teuthology.orchestra.run.smithi160.stdout: "active_change": "2020-05-24 13:49:45.331777", 2020-05-24T13:50:20.722 INFO:teuthology.orchestra.run.smithi160.stdout: "available": true, 2020-05-24T13:50:20.722 INFO:teuthology.orchestra.run.smithi160.stdout: "standbys": [], 2020-05-24T13:50:20.722 INFO:teuthology.orchestra.run.smithi160.stdout: "modules": [ 2020-05-24T13:50:20.722 INFO:teuthology.orchestra.run.smithi160.stdout: "iostat", 2020-05-24T13:50:20.722 INFO:teuthology.orchestra.run.smithi160.stdout: "restful" 2020-05-24T13:50:20.723 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2020-05-24T13:50:20.723 INFO:teuthology.orchestra.run.smithi160.stdout: "available_modules": [ 2020-05-24T13:50:20.723 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.723 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "alerts", 2020-05-24T13:50:20.723 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.724 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.724 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.724 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2020-05-24T13:50:20.724 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2020-05-24T13:50:20.724 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.724 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.725 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.725 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2020-05-24T13:50:20.725 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.725 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.725 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.726 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "How frequently to reexamine health status", 2020-05-24T13:50:20.726 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.726 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.726 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.727 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.727 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_destination": { 2020-05-24T13:50:20.727 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_destination", 2020-05-24T13:50:20.727 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.727 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.728 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.728 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.728 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.728 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.728 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.728 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Email address to send alerts to", 2020-05-24T13:50:20.729 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.729 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.729 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.730 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.730 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_from_name": { 2020-05-24T13:50:20.730 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_from_name", 2020-05-24T13:50:20.730 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.730 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.730 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.731 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "Ceph", 2020-05-24T13:50:20.731 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.731 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.731 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.731 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Email From: name", 2020-05-24T13:50:20.732 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.732 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.732 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.732 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.732 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_host": { 2020-05-24T13:50:20.732 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_host", 2020-05-24T13:50:20.733 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.733 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.733 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.733 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.733 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.734 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.734 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.734 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP server", 2020-05-24T13:50:20.734 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.734 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.734 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.735 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.735 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_password": { 2020-05-24T13:50:20.735 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_password", 2020-05-24T13:50:20.735 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.735 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.736 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.736 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.736 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.736 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.736 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.736 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Password to authenticate with", 2020-05-24T13:50:20.737 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.737 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.737 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.737 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.737 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_port": { 2020-05-24T13:50:20.737 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_port", 2020-05-24T13:50:20.738 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:20.738 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.738 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.738 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "465", 2020-05-24T13:50:20.738 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.739 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.739 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.739 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP port", 2020-05-24T13:50:20.739 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.740 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.740 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.740 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.740 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_sender": { 2020-05-24T13:50:20.740 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_sender", 2020-05-24T13:50:20.741 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.741 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.741 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.741 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.741 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.741 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.742 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.742 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "SMTP envelope sender", 2020-05-24T13:50:20.742 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.742 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.742 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.743 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.743 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_ssl": { 2020-05-24T13:50:20.743 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_ssl", 2020-05-24T13:50:20.743 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.743 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.743 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.744 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.744 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.744 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.744 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.744 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Use SSL to connect to SMTP server", 2020-05-24T13:50:20.744 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.745 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.745 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.745 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.745 INFO:teuthology.orchestra.run.smithi160.stdout: "smtp_user": { 2020-05-24T13:50:20.745 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "smtp_user", 2020-05-24T13:50:20.746 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.746 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.746 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.746 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.746 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.746 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.747 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.747 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "User to authenticate as", 2020-05-24T13:50:20.747 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.747 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.747 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.748 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.748 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.748 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.748 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.748 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ansible", 2020-05-24T13:50:20.748 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.749 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.749 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.749 INFO:teuthology.orchestra.run.smithi160.stdout: "password": { 2020-05-24T13:50:20.749 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "password", 2020-05-24T13:50:20.750 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.750 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.750 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.750 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.751 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.751 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.751 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.751 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.751 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.752 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.752 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.752 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.752 INFO:teuthology.orchestra.run.smithi160.stdout: "server_url": { 2020-05-24T13:50:20.752 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_url", 2020-05-24T13:50:20.753 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.753 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.753 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.753 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.753 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.754 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.754 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.754 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.754 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.754 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.754 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.755 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.755 INFO:teuthology.orchestra.run.smithi160.stdout: "username": { 2020-05-24T13:50:20.755 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "username", 2020-05-24T13:50:20.755 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.755 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.755 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.756 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.756 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.756 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.756 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.756 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.757 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.757 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.757 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.757 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.757 INFO:teuthology.orchestra.run.smithi160.stdout: "verify_server": { 2020-05-24T13:50:20.757 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "verify_server", 2020-05-24T13:50:20.758 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.758 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.758 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.758 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.758 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.758 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.759 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.759 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.759 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.759 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.759 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.760 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.760 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.760 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.760 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.761 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "balancer", 2020-05-24T13:50:20.761 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.761 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.761 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.761 INFO:teuthology.orchestra.run.smithi160.stdout: "active": { 2020-05-24T13:50:20.762 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "active", 2020-05-24T13:50:20.762 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.762 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.762 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.762 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2020-05-24T13:50:20.763 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.763 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.763 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.763 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "automatically balance PGs across cluster", 2020-05-24T13:50:20.764 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.764 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.764 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.764 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.765 INFO:teuthology.orchestra.run.smithi160.stdout: "begin_time": { 2020-05-24T13:50:20.765 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "begin_time", 2020-05-24T13:50:20.765 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.765 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.765 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.765 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0000", 2020-05-24T13:50:20.766 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.766 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.766 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.766 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "beginning time of day to automatically balance", 2020-05-24T13:50:20.766 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "This is a time of day in the format HHMM.", 2020-05-24T13:50:20.766 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.767 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.767 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.767 INFO:teuthology.orchestra.run.smithi160.stdout: "begin_weekday": { 2020-05-24T13:50:20.767 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "begin_weekday", 2020-05-24T13:50:20.767 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2020-05-24T13:50:20.768 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.768 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.768 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2020-05-24T13:50:20.768 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0", 2020-05-24T13:50:20.768 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "7", 2020-05-24T13:50:20.768 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.769 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2020-05-24T13:50:20.769 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-05-24T13:50:20.769 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.769 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.769 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.770 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_max_iterations": { 2020-05-24T13:50:20.770 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_max_iterations", 2020-05-24T13:50:20.770 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2020-05-24T13:50:20.770 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.771 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.771 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "25", 2020-05-24T13:50:20.771 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2020-05-24T13:50:20.771 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "250", 2020-05-24T13:50:20.771 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.772 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "maximum number of iterations to attempt optimization", 2020-05-24T13:50:20.772 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.772 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.772 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.772 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.772 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_metrics": { 2020-05-24T13:50:20.773 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_metrics", 2020-05-24T13:50:20.773 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.773 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.773 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.773 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "pgs,objects,bytes", 2020-05-24T13:50:20.773 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.774 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.774 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.774 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "metrics with which to calculate OSD utilization", 2020-05-24T13:50:20.774 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2020-05-24T13:50:20.774 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.775 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.775 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.775 INFO:teuthology.orchestra.run.smithi160.stdout: "crush_compat_step": { 2020-05-24T13:50:20.775 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crush_compat_step", 2020-05-24T13:50:20.775 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2020-05-24T13:50:20.776 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.776 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.776 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0.5", 2020-05-24T13:50:20.776 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0.001", 2020-05-24T13:50:20.776 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "0.999", 2020-05-24T13:50:20.776 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.777 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "aggressiveness of optimization", 2020-05-24T13:50:20.777 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2020-05-24T13:50:20.777 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.777 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.777 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.778 INFO:teuthology.orchestra.run.smithi160.stdout: "end_time": { 2020-05-24T13:50:20.778 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "end_time", 2020-05-24T13:50:20.778 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.778 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.778 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.778 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "2400", 2020-05-24T13:50:20.779 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.779 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.779 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.779 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "ending time of day to automatically balance", 2020-05-24T13:50:20.779 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "This is a time of day in the format HHMM.", 2020-05-24T13:50:20.780 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.780 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.780 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.780 INFO:teuthology.orchestra.run.smithi160.stdout: "end_weekday": { 2020-05-24T13:50:20.781 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "end_weekday", 2020-05-24T13:50:20.781 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2020-05-24T13:50:20.781 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.781 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.781 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "7", 2020-05-24T13:50:20.781 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "0", 2020-05-24T13:50:20.782 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "7", 2020-05-24T13:50:20.782 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.782 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2020-05-24T13:50:20.782 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2020-05-24T13:50:20.782 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.783 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.783 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.783 INFO:teuthology.orchestra.run.smithi160.stdout: "min_score": { 2020-05-24T13:50:20.783 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "min_score", 2020-05-24T13:50:20.783 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2020-05-24T13:50:20.783 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.784 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.784 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "0", 2020-05-24T13:50:20.784 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.784 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.784 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.785 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "minimum score, below which no optimization is attempted", 2020-05-24T13:50:20.785 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.785 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.785 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.785 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.785 INFO:teuthology.orchestra.run.smithi160.stdout: "mode": { 2020-05-24T13:50:20.786 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mode", 2020-05-24T13:50:20.788 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.788 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.788 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.788 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "none", 2020-05-24T13:50:20.789 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.789 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.789 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2020-05-24T13:50:20.789 INFO:teuthology.orchestra.run.smithi160.stdout: "crush-compat", 2020-05-24T13:50:20.789 INFO:teuthology.orchestra.run.smithi160.stdout: "none", 2020-05-24T13:50:20.789 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap" 2020-05-24T13:50:20.790 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2020-05-24T13:50:20.790 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Balancer mode", 2020-05-24T13:50:20.790 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.790 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.790 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.791 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.791 INFO:teuthology.orchestra.run.smithi160.stdout: "pool_ids": { 2020-05-24T13:50:20.791 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pool_ids", 2020-05-24T13:50:20.791 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.792 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.792 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.792 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.792 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.792 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.793 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.793 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "pools which the automatic balancing will be limited to", 2020-05-24T13:50:20.793 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.793 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.793 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.793 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.794 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2020-05-24T13:50:20.794 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2020-05-24T13:50:20.794 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:20.794 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.794 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.795 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2020-05-24T13:50:20.795 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.795 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.795 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.795 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to wake up and attempt optimization", 2020-05-24T13:50:20.795 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.796 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.796 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.796 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.796 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap_max_deviation": { 2020-05-24T13:50:20.796 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "upmap_max_deviation", 2020-05-24T13:50:20.796 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:20.797 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.797 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.797 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2020-05-24T13:50:20.797 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "1", 2020-05-24T13:50:20.797 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.798 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.798 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "deviation below which no optimization is attempted", 2020-05-24T13:50:20.798 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2020-05-24T13:50:20.798 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.798 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.798 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.799 INFO:teuthology.orchestra.run.smithi160.stdout: "upmap_max_iterations": { 2020-05-24T13:50:20.799 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "upmap_max_iterations", 2020-05-24T13:50:20.799 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "uint", 2020-05-24T13:50:20.799 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.799 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.800 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10", 2020-05-24T13:50:20.800 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.800 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.800 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.800 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "maximum upmap optimization iterations", 2020-05-24T13:50:20.800 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.801 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.801 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.801 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.801 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.802 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.802 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.802 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crash", 2020-05-24T13:50:20.802 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.803 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.803 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.803 INFO:teuthology.orchestra.run.smithi160.stdout: "retain_interval": { 2020-05-24T13:50:20.803 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "retain_interval", 2020-05-24T13:50:20.803 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:20.803 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.804 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.804 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "31536000", 2020-05-24T13:50:20.804 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.804 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.804 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.805 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long to retain crashes before pruning them", 2020-05-24T13:50:20.805 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.805 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.805 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.805 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.805 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_recent_interval": { 2020-05-24T13:50:20.806 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_recent_interval", 2020-05-24T13:50:20.806 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:20.806 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.806 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.806 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1209600", 2020-05-24T13:50:20.806 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.807 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.807 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.807 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "time interval in which to warn about recent crashes", 2020-05-24T13:50:20.807 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.807 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.808 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.808 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.808 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.808 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.808 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.808 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "dashboard", 2020-05-24T13:50:20.809 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.809 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.809 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.809 INFO:teuthology.orchestra.run.smithi160.stdout: "ALERTMANAGER_API_HOST": { 2020-05-24T13:50:20.809 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ALERTMANAGER_API_HOST", 2020-05-24T13:50:20.810 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.810 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.810 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.810 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.810 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.810 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.811 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.811 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.811 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.811 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.812 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.812 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.812 INFO:teuthology.orchestra.run.smithi160.stdout: "AUDIT_API_ENABLED": { 2020-05-24T13:50:20.812 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "AUDIT_API_ENABLED", 2020-05-24T13:50:20.812 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.813 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.813 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.813 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2020-05-24T13:50:20.813 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.813 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.813 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.814 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.814 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.814 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.814 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.814 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.814 INFO:teuthology.orchestra.run.smithi160.stdout: "AUDIT_API_LOG_PAYLOAD": { 2020-05-24T13:50:20.815 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2020-05-24T13:50:20.815 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.815 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.815 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.815 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.816 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.816 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.816 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.816 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.816 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.816 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.817 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.817 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.817 INFO:teuthology.orchestra.run.smithi160.stdout: "ENABLE_BROWSABLE_API": { 2020-05-24T13:50:20.817 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ENABLE_BROWSABLE_API", 2020-05-24T13:50:20.817 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.818 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.818 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.818 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.818 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.818 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.818 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.819 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.819 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.819 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.819 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.819 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.819 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_cephfs": { 2020-05-24T13:50:20.820 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_cephfs", 2020-05-24T13:50:20.820 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.820 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.820 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.820 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.821 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.821 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.821 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.821 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.821 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.822 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.822 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.822 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.822 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_iscsi": { 2020-05-24T13:50:20.823 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_iscsi", 2020-05-24T13:50:20.823 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.823 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.823 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.823 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.823 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.824 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.824 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.824 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.824 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.824 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.824 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.825 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.825 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_mirroring": { 2020-05-24T13:50:20.825 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_mirroring", 2020-05-24T13:50:20.825 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.825 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.826 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.827 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.827 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.827 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.827 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.828 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.828 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_rbd": { 2020-05-24T13:50:20.828 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_rbd", 2020-05-24T13:50:20.828 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.828 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.828 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.829 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.829 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.829 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.829 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.829 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.829 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.830 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.830 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.830 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.830 INFO:teuthology.orchestra.run.smithi160.stdout: "FEATURE_TOGGLE_rgw": { 2020-05-24T13:50:20.830 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "FEATURE_TOGGLE_rgw", 2020-05-24T13:50:20.831 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.831 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.831 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.831 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.831 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.831 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.832 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.832 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.832 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.832 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.833 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.833 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.833 INFO:teuthology.orchestra.run.smithi160.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2020-05-24T13:50:20.833 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2020-05-24T13:50:20.833 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.834 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.834 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.834 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.834 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.834 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.834 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.835 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.835 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.835 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.835 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.835 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.836 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_PASSWORD": { 2020-05-24T13:50:20.836 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_PASSWORD", 2020-05-24T13:50:20.836 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.836 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.836 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.836 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2020-05-24T13:50:20.837 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.837 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.837 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.837 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.837 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.837 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.838 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.838 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.838 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_SSL_VERIFY": { 2020-05-24T13:50:20.838 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2020-05-24T13:50:20.838 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.839 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.839 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.839 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.839 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.839 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.839 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.840 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.840 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.840 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.840 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.840 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.840 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_URL": { 2020-05-24T13:50:20.841 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_URL", 2020-05-24T13:50:20.841 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.841 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.841 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.841 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.842 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.842 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.842 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.842 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.843 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.843 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.843 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.843 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.843 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_API_USERNAME": { 2020-05-24T13:50:20.844 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_API_USERNAME", 2020-05-24T13:50:20.844 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.844 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.844 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.844 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2020-05-24T13:50:20.844 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.845 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.845 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.845 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.845 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.845 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.846 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.846 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.846 INFO:teuthology.orchestra.run.smithi160.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2020-05-24T13:50:20.846 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2020-05-24T13:50:20.846 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.846 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.847 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.847 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2020-05-24T13:50:20.847 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.847 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.847 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.847 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.848 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.848 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.848 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.848 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.848 INFO:teuthology.orchestra.run.smithi160.stdout: "ISCSI_API_SSL_VERIFICATION": { 2020-05-24T13:50:20.849 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2020-05-24T13:50:20.849 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.849 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.849 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.849 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.849 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.850 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.850 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.850 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.850 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.850 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.851 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.851 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.851 INFO:teuthology.orchestra.run.smithi160.stdout: "PROMETHEUS_API_HOST": { 2020-05-24T13:50:20.851 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "PROMETHEUS_API_HOST", 2020-05-24T13:50:20.851 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.852 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.852 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.852 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.852 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.853 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.853 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.853 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.853 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.853 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.854 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.854 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.854 INFO:teuthology.orchestra.run.smithi160.stdout: "REST_REQUESTS_TIMEOUT": { 2020-05-24T13:50:20.854 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "REST_REQUESTS_TIMEOUT", 2020-05-24T13:50:20.854 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:20.855 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.855 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.855 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "45", 2020-05-24T13:50:20.855 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.855 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.855 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.856 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.856 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.856 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.856 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.856 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.856 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_ACCESS_KEY": { 2020-05-24T13:50:20.857 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_ACCESS_KEY", 2020-05-24T13:50:20.857 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.857 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.857 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.857 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.858 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.858 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.858 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.858 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.858 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.859 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.859 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.859 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.859 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_ADMIN_RESOURCE": { 2020-05-24T13:50:20.859 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2020-05-24T13:50:20.860 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.860 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.860 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.860 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "admin", 2020-05-24T13:50:20.861 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.861 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.862 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.862 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.862 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.863 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.863 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.863 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.864 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_HOST": { 2020-05-24T13:50:20.864 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_HOST", 2020-05-24T13:50:20.865 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.865 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.866 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.866 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.866 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.867 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.867 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.867 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.868 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.868 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.869 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.869 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.869 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_PORT": { 2020-05-24T13:50:20.870 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_PORT", 2020-05-24T13:50:20.873 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:20.873 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.873 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.873 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "80", 2020-05-24T13:50:20.873 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.874 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.874 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.874 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.874 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.874 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.875 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.875 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.875 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SCHEME": { 2020-05-24T13:50:20.875 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SCHEME", 2020-05-24T13:50:20.875 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.875 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.876 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.876 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "http", 2020-05-24T13:50:20.876 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.876 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.876 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.877 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.877 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.877 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.877 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.877 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.877 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SECRET_KEY": { 2020-05-24T13:50:20.878 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SECRET_KEY", 2020-05-24T13:50:20.878 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.878 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.878 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.878 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.879 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.879 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.879 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.879 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.879 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.879 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.880 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.880 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.880 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_SSL_VERIFY": { 2020-05-24T13:50:20.880 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_SSL_VERIFY", 2020-05-24T13:50:20.880 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.880 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.881 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.881 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.881 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.881 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.881 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.882 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.882 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.882 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.882 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.882 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.883 INFO:teuthology.orchestra.run.smithi160.stdout: "RGW_API_USER_ID": { 2020-05-24T13:50:20.883 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "RGW_API_USER_ID", 2020-05-24T13:50:20.883 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.883 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.883 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.883 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.884 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.884 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.884 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.884 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.884 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.885 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.885 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.885 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.885 INFO:teuthology.orchestra.run.smithi160.stdout: "crt_file": { 2020-05-24T13:50:20.885 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "crt_file", 2020-05-24T13:50:20.886 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.886 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.886 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.886 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.886 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.887 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.887 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.887 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.887 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.888 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.888 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.888 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.888 INFO:teuthology.orchestra.run.smithi160.stdout: "debug": { 2020-05-24T13:50:20.888 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "debug", 2020-05-24T13:50:20.888 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.889 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.889 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.889 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2020-05-24T13:50:20.889 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.889 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.890 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.890 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Enable/disable debug options", 2020-05-24T13:50:20.890 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.890 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.890 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.890 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.891 INFO:teuthology.orchestra.run.smithi160.stdout: "jwt_token_ttl": { 2020-05-24T13:50:20.891 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "jwt_token_ttl", 2020-05-24T13:50:20.891 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:20.891 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.892 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.892 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "28800", 2020-05-24T13:50:20.892 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.892 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.892 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.893 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.893 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.893 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.893 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.893 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.893 INFO:teuthology.orchestra.run.smithi160.stdout: "key_file": { 2020-05-24T13:50:20.894 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "key_file", 2020-05-24T13:50:20.894 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.894 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.894 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.894 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.895 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.895 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.895 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.895 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.895 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.895 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.896 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.896 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.896 INFO:teuthology.orchestra.run.smithi160.stdout: "password": { 2020-05-24T13:50:20.896 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "password", 2020-05-24T13:50:20.896 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.896 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.897 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.897 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.897 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.897 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.897 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.898 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.898 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.898 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.898 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.898 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.898 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2020-05-24T13:50:20.899 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2020-05-24T13:50:20.899 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.899 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.899 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.899 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "::", 2020-05-24T13:50:20.899 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.900 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.900 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.900 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.900 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.900 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.901 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.901 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.901 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2020-05-24T13:50:20.901 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2020-05-24T13:50:20.901 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:20.901 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.902 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.902 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8080", 2020-05-24T13:50:20.902 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.902 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.902 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.903 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.903 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.903 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.903 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.903 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.903 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl": { 2020-05-24T13:50:20.904 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl", 2020-05-24T13:50:20.904 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.904 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.904 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.904 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.905 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.905 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.905 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.905 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.905 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.906 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.906 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.906 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.906 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl_server_port": { 2020-05-24T13:50:20.906 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl_server_port", 2020-05-24T13:50:20.907 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:20.907 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.907 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.907 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8443", 2020-05-24T13:50:20.907 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.907 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.908 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.908 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.908 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.908 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.909 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.909 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.909 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_behaviour": { 2020-05-24T13:50:20.909 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_behaviour", 2020-05-24T13:50:20.909 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.910 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.910 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.910 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "redirect", 2020-05-24T13:50:20.910 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.910 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.910 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [ 2020-05-24T13:50:20.911 INFO:teuthology.orchestra.run.smithi160.stdout: "error", 2020-05-24T13:50:20.911 INFO:teuthology.orchestra.run.smithi160.stdout: "redirect" 2020-05-24T13:50:20.911 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2020-05-24T13:50:20.911 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.911 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.912 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.912 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.912 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.912 INFO:teuthology.orchestra.run.smithi160.stdout: "standby_error_status_code": { 2020-05-24T13:50:20.912 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "standby_error_status_code", 2020-05-24T13:50:20.912 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:20.913 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.913 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.913 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "500", 2020-05-24T13:50:20.913 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "400", 2020-05-24T13:50:20.913 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "599", 2020-05-24T13:50:20.913 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.914 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.914 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.914 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.914 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.914 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.915 INFO:teuthology.orchestra.run.smithi160.stdout: "url_prefix": { 2020-05-24T13:50:20.915 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "url_prefix", 2020-05-24T13:50:20.915 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.915 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.915 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.915 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.916 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.916 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.916 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.916 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.916 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.917 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.917 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.917 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.917 INFO:teuthology.orchestra.run.smithi160.stdout: "username": { 2020-05-24T13:50:20.917 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "username", 2020-05-24T13:50:20.917 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.918 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.918 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.918 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.918 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.918 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.918 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.919 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.919 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.919 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.919 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.919 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.920 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.920 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.923 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.923 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "deepsea", 2020-05-24T13:50:20.923 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.923 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.923 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.924 INFO:teuthology.orchestra.run.smithi160.stdout: "salt_api_eauth": { 2020-05-24T13:50:20.924 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "salt_api_eauth", 2020-05-24T13:50:20.924 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.924 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.924 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.925 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "sharedsecret", 2020-05-24T13:50:20.925 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.925 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.925 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.925 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.925 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.926 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.926 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.926 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.926 INFO:teuthology.orchestra.run.smithi160.stdout: "salt_api_password": { 2020-05-24T13:50:20.926 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "salt_api_password", 2020-05-24T13:50:20.927 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.927 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.927 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.927 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.927 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.927 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.928 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.928 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.928 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.928 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.928 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.928 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.929 INFO:teuthology.orchestra.run.smithi160.stdout: "salt_api_url": { 2020-05-24T13:50:20.929 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "salt_api_url", 2020-05-24T13:50:20.929 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.929 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.929 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.930 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.930 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.930 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.930 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.930 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.930 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.931 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.931 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.931 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.931 INFO:teuthology.orchestra.run.smithi160.stdout: "salt_api_username": { 2020-05-24T13:50:20.931 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "salt_api_username", 2020-05-24T13:50:20.932 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.932 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.932 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.932 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.932 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.932 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.933 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.933 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.933 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.933 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.933 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.933 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.934 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.934 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.934 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.934 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "devicehealth", 2020-05-24T13:50:20.934 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.935 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.935 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.935 INFO:teuthology.orchestra.run.smithi160.stdout: "enable_monitoring": { 2020-05-24T13:50:20.935 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enable_monitoring", 2020-05-24T13:50:20.935 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.935 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.936 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.936 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2020-05-24T13:50:20.936 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.936 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.936 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.937 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "monitor device health metrics", 2020-05-24T13:50:20.937 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.937 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.937 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.937 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.937 INFO:teuthology.orchestra.run.smithi160.stdout: "mark_out_threshold": { 2020-05-24T13:50:20.938 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "mark_out_threshold", 2020-05-24T13:50:20.938 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:20.938 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.938 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.938 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "2419200", 2020-05-24T13:50:20.939 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.939 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.939 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.939 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "automatically mark OSD if it may fail before this long", 2020-05-24T13:50:20.939 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.939 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.940 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.940 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.940 INFO:teuthology.orchestra.run.smithi160.stdout: "pool_name": { 2020-05-24T13:50:20.940 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pool_name", 2020-05-24T13:50:20.940 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.940 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.941 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.941 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "device_health_metrics", 2020-05-24T13:50:20.941 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.941 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.941 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.942 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "name of pool in which to store device health metrics", 2020-05-24T13:50:20.942 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.942 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.942 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.942 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.942 INFO:teuthology.orchestra.run.smithi160.stdout: "retention_period": { 2020-05-24T13:50:20.943 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "retention_period", 2020-05-24T13:50:20.943 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:20.943 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.943 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.943 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15552000", 2020-05-24T13:50:20.944 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.944 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.944 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.944 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how long to retain device health metrics", 2020-05-24T13:50:20.944 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.944 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.945 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.945 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.945 INFO:teuthology.orchestra.run.smithi160.stdout: "scrape_frequency": { 2020-05-24T13:50:20.945 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "scrape_frequency", 2020-05-24T13:50:20.945 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:20.945 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.946 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.946 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "86400", 2020-05-24T13:50:20.946 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.946 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.946 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.947 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to scrape device health metrics", 2020-05-24T13:50:20.947 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.947 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.947 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.947 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.947 INFO:teuthology.orchestra.run.smithi160.stdout: "self_heal": { 2020-05-24T13:50:20.948 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "self_heal", 2020-05-24T13:50:20.948 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:20.948 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.948 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.948 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:20.948 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.949 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.949 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.949 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "preemptively heal cluster around devices that may fail", 2020-05-24T13:50:20.949 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.949 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.950 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.950 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.950 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2020-05-24T13:50:20.950 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2020-05-24T13:50:20.950 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:20.950 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.951 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.951 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2020-05-24T13:50:20.951 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.951 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.951 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.952 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to wake up and check device health", 2020-05-24T13:50:20.952 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.952 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.952 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.952 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.953 INFO:teuthology.orchestra.run.smithi160.stdout: "warn_threshold": { 2020-05-24T13:50:20.953 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "warn_threshold", 2020-05-24T13:50:20.953 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:20.953 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.953 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:20.953 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "7257600", 2020-05-24T13:50:20.954 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.954 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.954 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.954 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "raise health warning if OSD may fail before this long", 2020-05-24T13:50:20.954 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.955 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.955 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.955 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.955 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.955 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.955 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.956 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_cloud", 2020-05-24T13:50:20.956 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.956 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.956 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.956 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_cert_context": { 2020-05-24T13:50:20.956 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_cert_context", 2020-05-24T13:50:20.957 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.957 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.957 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.957 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.957 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.958 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.958 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.958 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.958 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.959 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.959 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.959 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.959 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_default_authority": { 2020-05-24T13:50:20.959 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_default_authority", 2020-05-24T13:50:20.959 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.960 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.960 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.960 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "localhost", 2020-05-24T13:50:20.960 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.960 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.961 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.961 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.961 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.961 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.961 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.961 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.962 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_password": { 2020-05-24T13:50:20.962 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_password", 2020-05-24T13:50:20.962 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.962 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.962 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.963 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.963 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.963 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.963 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.963 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.963 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.964 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.965 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.965 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.965 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_port": { 2020-05-24T13:50:20.965 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_port", 2020-05-24T13:50:20.966 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.966 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.966 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.966 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "31400", 2020-05-24T13:50:20.966 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.967 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.967 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.967 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.967 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.967 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.968 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.968 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.968 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_retrieve_prediction_interval": { 2020-05-24T13:50:20.968 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_retrieve_prediction_interval", 2020-05-24T13:50:20.968 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.968 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.969 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.969 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "43200", 2020-05-24T13:50:20.969 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.969 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.969 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.970 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.970 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.970 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.970 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.970 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.971 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_server": { 2020-05-24T13:50:20.971 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_server", 2020-05-24T13:50:20.971 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.971 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.971 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.971 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.972 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.972 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.972 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.972 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.972 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.973 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.973 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.973 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.973 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_ssl_target_name_override": { 2020-05-24T13:50:20.973 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_ssl_target_name_override", 2020-05-24T13:50:20.973 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.974 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.974 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.974 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "localhost", 2020-05-24T13:50:20.974 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.974 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.975 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.975 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.975 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.975 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.975 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.975 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.976 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_upload_metrics_interval": { 2020-05-24T13:50:20.976 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_upload_metrics_interval", 2020-05-24T13:50:20.976 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.976 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.976 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.977 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2020-05-24T13:50:20.977 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.977 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.977 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.977 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.977 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.978 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.978 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.978 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.978 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_upload_smart_interval": { 2020-05-24T13:50:20.978 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_upload_smart_interval", 2020-05-24T13:50:20.979 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.979 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.979 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.979 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "43200", 2020-05-24T13:50:20.979 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.979 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.980 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.980 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.980 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.980 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.980 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.980 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.981 INFO:teuthology.orchestra.run.smithi160.stdout: "diskprediction_user": { 2020-05-24T13:50:20.981 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_user", 2020-05-24T13:50:20.981 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.981 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.981 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.982 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:20.982 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.982 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.982 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.982 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.983 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.983 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.983 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.983 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.983 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2020-05-24T13:50:20.983 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2020-05-24T13:50:20.984 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.984 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.984 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.984 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2020-05-24T13:50:20.984 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.984 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.985 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.985 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.985 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.985 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.985 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.986 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.986 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.986 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.986 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.986 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "diskprediction_local", 2020-05-24T13:50:20.986 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:20.987 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:20.987 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.987 INFO:teuthology.orchestra.run.smithi160.stdout: "predict_interval": { 2020-05-24T13:50:20.987 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "predict_interval", 2020-05-24T13:50:20.987 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.988 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.988 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.988 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "86400", 2020-05-24T13:50:20.988 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.988 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.988 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.989 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.989 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.989 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.989 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.989 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.990 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2020-05-24T13:50:20.990 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2020-05-24T13:50:20.990 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.990 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.990 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.990 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "600", 2020-05-24T13:50:20.991 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.991 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.991 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.991 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.991 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.992 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.992 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.992 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.992 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:20.992 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.992 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:20.993 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "influx", 2020-05-24T13:50:20.993 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": false, 2020-05-24T13:50:20.993 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "influxdb python module not found", 2020-05-24T13:50:20.993 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:20.993 INFO:teuthology.orchestra.run.smithi160.stdout: "batch_size": { 2020-05-24T13:50:20.994 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "batch_size", 2020-05-24T13:50:20.994 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.994 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.994 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.994 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5000", 2020-05-24T13:50:20.994 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.995 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.995 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.995 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.995 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.995 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.996 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.996 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.996 INFO:teuthology.orchestra.run.smithi160.stdout: "database": { 2020-05-24T13:50:20.996 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "database", 2020-05-24T13:50:20.996 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.996 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.997 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.997 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "ceph", 2020-05-24T13:50:20.997 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:20.997 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:20.997 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:20.998 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:20.998 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:20.998 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:20.998 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:20.998 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:20.998 INFO:teuthology.orchestra.run.smithi160.stdout: "hostname": { 2020-05-24T13:50:20.999 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "hostname", 2020-05-24T13:50:20.999 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:20.999 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:20.999 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:20.999 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "None", 2020-05-24T13:50:20.999 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.000 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.000 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.000 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.000 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.000 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.001 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.001 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.001 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2020-05-24T13:50:21.001 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2020-05-24T13:50:21.001 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.001 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.002 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.002 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "30", 2020-05-24T13:50:21.002 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.002 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.002 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.003 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.003 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.003 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.003 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.003 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.003 INFO:teuthology.orchestra.run.smithi160.stdout: "password": { 2020-05-24T13:50:21.004 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "password", 2020-05-24T13:50:21.004 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.004 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.004 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.004 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "None", 2020-05-24T13:50:21.004 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.005 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.005 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.005 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.005 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.005 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.006 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.006 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.006 INFO:teuthology.orchestra.run.smithi160.stdout: "port": { 2020-05-24T13:50:21.006 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "port", 2020-05-24T13:50:21.006 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.006 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.007 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.007 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "8086", 2020-05-24T13:50:21.007 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.007 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.007 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.008 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.008 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.008 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.008 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.008 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.008 INFO:teuthology.orchestra.run.smithi160.stdout: "ssl": { 2020-05-24T13:50:21.009 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssl", 2020-05-24T13:50:21.009 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.009 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.009 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.009 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "false", 2020-05-24T13:50:21.010 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.010 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.010 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.010 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.010 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.010 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.011 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.011 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.011 INFO:teuthology.orchestra.run.smithi160.stdout: "threads": { 2020-05-24T13:50:21.011 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "threads", 2020-05-24T13:50:21.011 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.011 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.012 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.012 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2020-05-24T13:50:21.012 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.012 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.012 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.013 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.013 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.013 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.013 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.013 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.013 INFO:teuthology.orchestra.run.smithi160.stdout: "username": { 2020-05-24T13:50:21.014 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "username", 2020-05-24T13:50:21.014 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.014 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.014 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.014 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "None", 2020-05-24T13:50:21.015 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.015 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.015 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.015 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.015 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.015 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.016 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.016 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.016 INFO:teuthology.orchestra.run.smithi160.stdout: "verify_ssl": { 2020-05-24T13:50:21.016 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "verify_ssl", 2020-05-24T13:50:21.016 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.016 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.017 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.017 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "true", 2020-05-24T13:50:21.017 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.017 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.017 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.018 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.018 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.018 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.018 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.018 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.018 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.019 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.019 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.019 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "insights", 2020-05-24T13:50:21.019 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.019 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.020 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": {} 2020-05-24T13:50:21.020 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.020 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.020 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "iostat", 2020-05-24T13:50:21.020 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.020 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.021 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": {} 2020-05-24T13:50:21.021 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.021 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.021 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "localpool", 2020-05-24T13:50:21.021 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.022 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.022 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.022 INFO:teuthology.orchestra.run.smithi160.stdout: "failure_domain": { 2020-05-24T13:50:21.022 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "failure_domain", 2020-05-24T13:50:21.023 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.023 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.023 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:21.023 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "host", 2020-05-24T13:50:21.023 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.024 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.024 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.024 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "failure domain for any created local pool", 2020-05-24T13:50:21.024 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.024 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.025 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.025 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.025 INFO:teuthology.orchestra.run.smithi160.stdout: "min_size": { 2020-05-24T13:50:21.025 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "min_size", 2020-05-24T13:50:21.025 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:21.026 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.026 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:21.026 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.026 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.026 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.026 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.027 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default min_size for any created local pool", 2020-05-24T13:50:21.027 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.027 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.027 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.027 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.027 INFO:teuthology.orchestra.run.smithi160.stdout: "num_rep": { 2020-05-24T13:50:21.028 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "num_rep", 2020-05-24T13:50:21.028 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:21.028 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.028 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:21.028 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "3", 2020-05-24T13:50:21.029 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.029 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.029 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.029 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default replica count for any created local pool", 2020-05-24T13:50:21.030 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.030 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.030 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.030 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.030 INFO:teuthology.orchestra.run.smithi160.stdout: "pg_num": { 2020-05-24T13:50:21.030 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pg_num", 2020-05-24T13:50:21.031 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:21.031 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.031 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:21.031 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "128", 2020-05-24T13:50:21.031 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.031 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.032 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.032 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "default pg_num for any created local pool", 2020-05-24T13:50:21.032 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.032 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.033 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.033 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.033 INFO:teuthology.orchestra.run.smithi160.stdout: "prefix": { 2020-05-24T13:50:21.033 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prefix", 2020-05-24T13:50:21.033 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.034 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.034 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:21.034 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.034 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.034 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.034 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.035 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "name prefix for any created local pool", 2020-05-24T13:50:21.035 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.035 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.035 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.035 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.036 INFO:teuthology.orchestra.run.smithi160.stdout: "subtree": { 2020-05-24T13:50:21.036 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "subtree", 2020-05-24T13:50:21.036 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.036 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.036 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:21.036 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "rack", 2020-05-24T13:50:21.037 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.037 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.037 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.037 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "CRUSH level for which to create a local pool", 2020-05-24T13:50:21.037 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.038 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.038 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.038 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.038 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.038 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.039 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.039 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "orchestrator_cli", 2020-05-24T13:50:21.039 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.039 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.039 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.039 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator": { 2020-05-24T13:50:21.040 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "orchestrator", 2020-05-24T13:50:21.040 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.040 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.040 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.040 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.041 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.041 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.041 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.041 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.041 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.041 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.042 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.042 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.042 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.042 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.042 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.042 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "pg_autoscaler", 2020-05-24T13:50:21.043 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.043 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.043 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.043 INFO:teuthology.orchestra.run.smithi160.stdout: "sleep_interval": { 2020-05-24T13:50:21.043 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "sleep_interval", 2020-05-24T13:50:21.044 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.044 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.044 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.044 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2020-05-24T13:50:21.044 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.044 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.045 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.045 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.045 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.045 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.045 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.045 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.046 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.046 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.046 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.046 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "progress", 2020-05-24T13:50:21.046 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.047 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.047 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.047 INFO:teuthology.orchestra.run.smithi160.stdout: "max_completed_events": { 2020-05-24T13:50:21.047 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "max_completed_events", 2020-05-24T13:50:21.047 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:21.047 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.048 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:21.048 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "50", 2020-05-24T13:50:21.048 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.048 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.048 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.048 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "number of past completed events to remember", 2020-05-24T13:50:21.049 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.049 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.049 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.049 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.049 INFO:teuthology.orchestra.run.smithi160.stdout: "persist_interval": { 2020-05-24T13:50:21.050 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "persist_interval", 2020-05-24T13:50:21.050 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:21.050 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.050 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 1, 2020-05-24T13:50:21.050 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "5", 2020-05-24T13:50:21.050 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.051 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.051 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.051 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "how frequently to persist completed events", 2020-05-24T13:50:21.051 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.051 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.051 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.052 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.052 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.052 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.052 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.052 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "prometheus", 2020-05-24T13:50:21.053 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.053 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.053 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.053 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_stats_pools": { 2020-05-24T13:50:21.053 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_stats_pools", 2020-05-24T13:50:21.053 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.054 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.054 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.054 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.054 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.054 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.055 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.055 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.055 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.055 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.055 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.055 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.056 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_stats_pools_refresh_interval": { 2020-05-24T13:50:21.056 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_stats_pools_refresh_interval", 2020-05-24T13:50:21.056 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.056 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.056 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.056 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.057 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.057 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.057 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.057 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.057 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.058 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.058 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.058 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.058 INFO:teuthology.orchestra.run.smithi160.stdout: "scrape_interval": { 2020-05-24T13:50:21.058 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "scrape_interval", 2020-05-24T13:50:21.058 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.059 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.059 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.059 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.059 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.059 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.059 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.060 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.060 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.060 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.060 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.060 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.061 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2020-05-24T13:50:21.061 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2020-05-24T13:50:21.061 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.061 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.061 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.062 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.062 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.062 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.062 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.062 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.063 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.063 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.063 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.063 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.063 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2020-05-24T13:50:21.064 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2020-05-24T13:50:21.064 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.064 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.064 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.064 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.065 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.065 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.065 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.065 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.065 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.065 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.066 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.066 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.066 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.066 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.066 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.066 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rbd_support", 2020-05-24T13:50:21.067 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.067 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.067 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": {} 2020-05-24T13:50:21.067 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.067 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.068 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "restful", 2020-05-24T13:50:21.068 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.068 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.068 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.068 INFO:teuthology.orchestra.run.smithi160.stdout: "key_file": { 2020-05-24T13:50:21.068 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "key_file", 2020-05-24T13:50:21.069 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.069 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.069 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.070 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.070 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.070 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.070 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.070 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.070 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.071 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.071 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.071 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.071 INFO:teuthology.orchestra.run.smithi160.stdout: "server_addr": { 2020-05-24T13:50:21.072 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_addr", 2020-05-24T13:50:21.072 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.072 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.072 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.072 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.073 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.073 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.073 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.073 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.073 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.073 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.074 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.074 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.074 INFO:teuthology.orchestra.run.smithi160.stdout: "server_port": { 2020-05-24T13:50:21.074 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "server_port", 2020-05-24T13:50:21.074 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.075 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.075 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.075 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.075 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.075 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.075 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.076 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.076 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.076 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.076 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.076 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.076 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.077 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.077 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.077 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rook", 2020-05-24T13:50:21.077 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.077 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.078 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": {} 2020-05-24T13:50:21.078 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.078 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.078 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "selftest", 2020-05-24T13:50:21.078 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.078 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.079 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.079 INFO:teuthology.orchestra.run.smithi160.stdout: "roption1": { 2020-05-24T13:50:21.079 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "roption1", 2020-05-24T13:50:21.079 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.079 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.080 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.080 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.080 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.080 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.081 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.081 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.081 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.081 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.081 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.082 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.082 INFO:teuthology.orchestra.run.smithi160.stdout: "roption2": { 2020-05-24T13:50:21.082 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "roption2", 2020-05-24T13:50:21.082 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.082 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.083 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.083 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "xyz", 2020-05-24T13:50:21.083 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.083 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.083 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.083 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.084 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.084 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.084 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.084 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.084 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption1": { 2020-05-24T13:50:21.084 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption1", 2020-05-24T13:50:21.085 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.085 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.085 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.085 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.085 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.086 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.086 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.086 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.086 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.086 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.086 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.087 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.087 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption2": { 2020-05-24T13:50:21.087 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption2", 2020-05-24T13:50:21.087 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:21.087 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.087 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.088 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.088 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.088 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.088 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.088 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.089 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.089 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.089 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.089 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.089 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption3": { 2020-05-24T13:50:21.089 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption3", 2020-05-24T13:50:21.090 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "float", 2020-05-24T13:50:21.090 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.090 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.090 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.090 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.090 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.091 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.091 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.091 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.091 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.091 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.092 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.092 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption4": { 2020-05-24T13:50:21.092 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption4", 2020-05-24T13:50:21.092 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.092 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.092 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.093 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.093 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.093 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.093 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.093 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.093 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.094 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.094 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.094 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.094 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption5": { 2020-05-24T13:50:21.094 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption5", 2020-05-24T13:50:21.095 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:21.095 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.095 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.095 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.095 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.095 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.096 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.096 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.096 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.096 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.096 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.097 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.097 INFO:teuthology.orchestra.run.smithi160.stdout: "rwoption6": { 2020-05-24T13:50:21.097 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "rwoption6", 2020-05-24T13:50:21.097 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:21.098 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.098 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.098 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:21.098 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.098 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.099 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.099 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.099 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.099 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.099 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.100 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.100 INFO:teuthology.orchestra.run.smithi160.stdout: "testkey": { 2020-05-24T13:50:21.100 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testkey", 2020-05-24T13:50:21.100 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.100 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.100 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.101 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.101 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.101 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.101 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.101 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.101 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.102 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.102 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.102 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.102 INFO:teuthology.orchestra.run.smithi160.stdout: "testlkey": { 2020-05-24T13:50:21.102 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testlkey", 2020-05-24T13:50:21.103 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.103 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.103 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.103 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.103 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.103 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.104 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.104 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.104 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.104 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.104 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.104 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.105 INFO:teuthology.orchestra.run.smithi160.stdout: "testnewline": { 2020-05-24T13:50:21.105 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "testnewline", 2020-05-24T13:50:21.105 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.105 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.105 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.106 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.106 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.106 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.106 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.106 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.106 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.107 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.107 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.107 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.107 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.107 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.107 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.108 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssh", 2020-05-24T13:50:21.108 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.108 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.108 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.108 INFO:teuthology.orchestra.run.smithi160.stdout: "inventory_cache_timeout_min": { 2020-05-24T13:50:21.108 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "inventory_cache_timeout_min", 2020-05-24T13:50:21.109 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.109 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.109 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.109 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.109 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.110 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.110 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.110 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.110 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.111 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.111 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.111 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.111 INFO:teuthology.orchestra.run.smithi160.stdout: "ssh_config_file": { 2020-05-24T13:50:21.111 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "ssh_config_file", 2020-05-24T13:50:21.112 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.112 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.112 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.112 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.113 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.113 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.113 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.113 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.113 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.114 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.114 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.114 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.114 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.115 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.115 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.115 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "status", 2020-05-24T13:50:21.115 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.115 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.116 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": {} 2020-05-24T13:50:21.116 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.116 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.117 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "telegraf", 2020-05-24T13:50:21.117 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.117 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.117 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.117 INFO:teuthology.orchestra.run.smithi160.stdout: "address": { 2020-05-24T13:50:21.118 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "address", 2020-05-24T13:50:21.118 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.118 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.118 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.118 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2020-05-24T13:50:21.119 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.119 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.119 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.119 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.119 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.120 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.120 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.120 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.120 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2020-05-24T13:50:21.120 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2020-05-24T13:50:21.120 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:21.121 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.121 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.121 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "15", 2020-05-24T13:50:21.121 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.121 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.122 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.122 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.122 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.122 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.122 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.122 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.123 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.123 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.123 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.123 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "telemetry", 2020-05-24T13:50:21.123 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.124 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.124 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.124 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_basic": { 2020-05-24T13:50:21.124 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_basic", 2020-05-24T13:50:21.124 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:21.125 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.125 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.125 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:21.125 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.125 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.126 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.126 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "Share basic cluster information (size, version)", 2020-05-24T13:50:21.126 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.126 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.126 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.126 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.127 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_crash": { 2020-05-24T13:50:21.127 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_crash", 2020-05-24T13:50:21.127 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:21.127 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.127 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.128 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:21.128 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.128 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.128 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.128 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.128 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.129 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.129 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.129 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.129 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_device": { 2020-05-24T13:50:21.129 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_device", 2020-05-24T13:50:21.130 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:21.130 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.130 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.130 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "True", 2020-05-24T13:50:21.130 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.131 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.131 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.131 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.131 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.131 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.132 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.132 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.132 INFO:teuthology.orchestra.run.smithi160.stdout: "channel_ident": { 2020-05-24T13:50:21.132 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "channel_ident", 2020-05-24T13:50:21.132 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:21.133 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.133 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.133 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2020-05-24T13:50:21.133 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.133 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.134 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.134 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.134 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.134 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.134 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.134 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.135 INFO:teuthology.orchestra.run.smithi160.stdout: "contact": { 2020-05-24T13:50:21.135 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "contact", 2020-05-24T13:50:21.135 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.135 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.135 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.136 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "None", 2020-05-24T13:50:21.136 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.136 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.136 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.136 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.136 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.137 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.137 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.137 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.137 INFO:teuthology.orchestra.run.smithi160.stdout: "description": { 2020-05-24T13:50:21.137 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "description", 2020-05-24T13:50:21.138 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.138 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.138 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.138 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "None", 2020-05-24T13:50:21.138 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.139 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.139 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.139 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.139 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.139 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.140 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.140 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.140 INFO:teuthology.orchestra.run.smithi160.stdout: "device_url": { 2020-05-24T13:50:21.140 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "device_url", 2020-05-24T13:50:21.140 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.140 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.141 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.141 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "https://telemetry.ceph.com/device", 2020-05-24T13:50:21.141 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.141 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.141 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.141 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.142 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.142 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.142 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.142 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.142 INFO:teuthology.orchestra.run.smithi160.stdout: "enabled": { 2020-05-24T13:50:21.143 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "enabled", 2020-05-24T13:50:21.143 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:21.143 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.143 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.143 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2020-05-24T13:50:21.143 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.144 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.144 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.144 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.144 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.144 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.144 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.145 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.145 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2020-05-24T13:50:21.145 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2020-05-24T13:50:21.145 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:21.145 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.146 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.146 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "24", 2020-05-24T13:50:21.146 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "8", 2020-05-24T13:50:21.146 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.146 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.146 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.147 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.147 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.147 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.147 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.147 INFO:teuthology.orchestra.run.smithi160.stdout: "last_opt_revision": { 2020-05-24T13:50:21.148 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "last_opt_revision", 2020-05-24T13:50:21.148 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:21.148 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.148 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.148 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "1", 2020-05-24T13:50:21.148 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.149 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.149 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.149 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.149 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.149 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.150 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.150 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.150 INFO:teuthology.orchestra.run.smithi160.stdout: "leaderboard": { 2020-05-24T13:50:21.150 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "leaderboard", 2020-05-24T13:50:21.150 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "bool", 2020-05-24T13:50:21.150 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.151 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.151 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "False", 2020-05-24T13:50:21.151 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.151 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.151 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.152 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.152 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.152 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.152 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.152 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.152 INFO:teuthology.orchestra.run.smithi160.stdout: "organization": { 2020-05-24T13:50:21.153 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "organization", 2020-05-24T13:50:21.153 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.153 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.153 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.153 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "None", 2020-05-24T13:50:21.153 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.154 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.154 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.154 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.154 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.154 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.155 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.155 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.155 INFO:teuthology.orchestra.run.smithi160.stdout: "proxy": { 2020-05-24T13:50:21.155 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "proxy", 2020-05-24T13:50:21.155 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.155 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.156 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.156 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "None", 2020-05-24T13:50:21.156 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.156 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.156 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.156 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.157 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.157 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.157 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.157 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.157 INFO:teuthology.orchestra.run.smithi160.stdout: "url": { 2020-05-24T13:50:21.158 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "url", 2020-05-24T13:50:21.158 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.158 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.158 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.158 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "https://telemetry.ceph.com/report", 2020-05-24T13:50:21.158 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.159 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.159 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.159 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.159 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.159 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.160 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.160 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.160 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.160 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.160 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.160 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "test_orchestrator", 2020-05-24T13:50:21.161 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.161 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.161 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": {} 2020-05-24T13:50:21.161 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.161 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.162 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "volumes", 2020-05-24T13:50:21.162 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.162 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.162 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": {} 2020-05-24T13:50:21.162 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.163 INFO:teuthology.orchestra.run.smithi160.stdout: { 2020-05-24T13:50:21.163 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix", 2020-05-24T13:50:21.163 INFO:teuthology.orchestra.run.smithi160.stdout: "can_run": true, 2020-05-24T13:50:21.163 INFO:teuthology.orchestra.run.smithi160.stdout: "error_string": "", 2020-05-24T13:50:21.163 INFO:teuthology.orchestra.run.smithi160.stdout: "module_options": { 2020-05-24T13:50:21.164 INFO:teuthology.orchestra.run.smithi160.stdout: "identifier": { 2020-05-24T13:50:21.164 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "identifier", 2020-05-24T13:50:21.164 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.164 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.165 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.165 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "", 2020-05-24T13:50:21.165 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.165 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.166 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.166 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.166 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.166 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.166 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.167 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.167 INFO:teuthology.orchestra.run.smithi160.stdout: "interval": { 2020-05-24T13:50:21.167 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "interval", 2020-05-24T13:50:21.167 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "secs", 2020-05-24T13:50:21.167 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.167 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.168 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "60", 2020-05-24T13:50:21.168 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.168 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.168 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.168 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.169 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.169 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.169 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.169 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.169 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_host": { 2020-05-24T13:50:21.170 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_host", 2020-05-24T13:50:21.170 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.170 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.170 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.170 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "None", 2020-05-24T13:50:21.171 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.171 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.171 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.172 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.172 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.172 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.172 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.172 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.173 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_port": { 2020-05-24T13:50:21.173 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_port", 2020-05-24T13:50:21.173 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "int", 2020-05-24T13:50:21.173 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.174 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.174 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "10051", 2020-05-24T13:50:21.174 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.174 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.174 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.174 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.175 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.175 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.175 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.175 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.175 INFO:teuthology.orchestra.run.smithi160.stdout: "zabbix_sender": { 2020-05-24T13:50:21.176 INFO:teuthology.orchestra.run.smithi160.stdout: "name": "zabbix_sender", 2020-05-24T13:50:21.176 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "str", 2020-05-24T13:50:21.176 INFO:teuthology.orchestra.run.smithi160.stdout: "level": "advanced", 2020-05-24T13:50:21.176 INFO:teuthology.orchestra.run.smithi160.stdout: "flags": 0, 2020-05-24T13:50:21.176 INFO:teuthology.orchestra.run.smithi160.stdout: "default_value": "/usr/bin/zabbix_sender", 2020-05-24T13:50:21.176 INFO:teuthology.orchestra.run.smithi160.stdout: "min": "", 2020-05-24T13:50:21.177 INFO:teuthology.orchestra.run.smithi160.stdout: "max": "", 2020-05-24T13:50:21.177 INFO:teuthology.orchestra.run.smithi160.stdout: "enum_allowed": [], 2020-05-24T13:50:21.177 INFO:teuthology.orchestra.run.smithi160.stdout: "desc": "", 2020-05-24T13:50:21.177 INFO:teuthology.orchestra.run.smithi160.stdout: "long_desc": "", 2020-05-24T13:50:21.177 INFO:teuthology.orchestra.run.smithi160.stdout: "tags": [], 2020-05-24T13:50:21.177 INFO:teuthology.orchestra.run.smithi160.stdout: "see_also": [] 2020-05-24T13:50:21.178 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.178 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.178 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.178 INFO:teuthology.orchestra.run.smithi160.stdout: ], 2020-05-24T13:50:21.178 INFO:teuthology.orchestra.run.smithi160.stdout: "services": {}, 2020-05-24T13:50:21.179 INFO:teuthology.orchestra.run.smithi160.stdout: "always_on_modules": { 2020-05-24T13:50:21.179 INFO:teuthology.orchestra.run.smithi160.stdout: "nautilus": [ 2020-05-24T13:50:21.179 INFO:teuthology.orchestra.run.smithi160.stdout: "balancer", 2020-05-24T13:50:21.179 INFO:teuthology.orchestra.run.smithi160.stdout: "crash", 2020-05-24T13:50:21.179 INFO:teuthology.orchestra.run.smithi160.stdout: "devicehealth", 2020-05-24T13:50:21.179 INFO:teuthology.orchestra.run.smithi160.stdout: "orchestrator_cli", 2020-05-24T13:50:21.180 INFO:teuthology.orchestra.run.smithi160.stdout: "progress", 2020-05-24T13:50:21.180 INFO:teuthology.orchestra.run.smithi160.stdout: "rbd_support", 2020-05-24T13:50:21.180 INFO:teuthology.orchestra.run.smithi160.stdout: "status", 2020-05-24T13:50:21.180 INFO:teuthology.orchestra.run.smithi160.stdout: "volumes" 2020-05-24T13:50:21.180 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2020-05-24T13:50:21.181 INFO:teuthology.orchestra.run.smithi160.stdout: } 2020-05-24T13:50:21.181 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.181 INFO:teuthology.orchestra.run.smithi160.stdout: "servicemap": { 2020-05-24T13:50:21.181 INFO:teuthology.orchestra.run.smithi160.stdout: "epoch": 1, 2020-05-24T13:50:21.182 INFO:teuthology.orchestra.run.smithi160.stdout: "modified": "0.000000", 2020-05-24T13:50:21.182 INFO:teuthology.orchestra.run.smithi160.stdout: "services": {} 2020-05-24T13:50:21.182 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2020-05-24T13:50:21.182 INFO:teuthology.orchestra.run.smithi160.stdout: "progress_events": {} 2020-05-24T13:50:21.182 INFO:teuthology.orchestra.run.smithi160.stdout:} 2020-05-24T13:50:21.183 INFO:tasks.ceph.ceph_manager.c2:{'fsid': 'e6d3bc87-1c02-4a97-acf9-0c849318394c', 'health': {'checks': {}, 'status': 'HEALTH_OK'}, 'election_epoch': 3, 'quorum': [0], 'quorum_names': ['a'], 'quorum_age': 40, 'monmap': {'epoch': 1, 'fsid': 'e6d3bc87-1c02-4a97-acf9-0c849318394c', 'modified': '2020-05-24 13:49:17.140145', 'created': '2020-05-24 13:49:17.140145', 'min_mon_release': 14, 'min_mon_release_name': 'nautilus', 'features': {'persistent': ['kraken', 'luminous', 'mimic', 'osdmap-prune', 'nautilus'], 'optional': []}, 'mons': [{'rank': 0, 'name': 'a', 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:3300', 'nonce': 0}, {'type': 'v1', 'addr': '172.21.15.160:6789', 'nonce': 0}]}, 'addr': '172.21.15.160:6789/0', 'public_addr': '172.21.15.160:6789/0'}]}, 'osdmap': {'osdmap': {'epoch': 13, 'num_osds': 3, 'num_up_osds': 3, 'num_in_osds': 3, 'num_remapped_pgs': 0}}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 8}], 'num_pgs': 8, 'num_pools': 1, 'num_objects': 0, 'data_bytes': 0, 'bytes_used': 3221594112, 'bytes_avail': 318900953088, 'bytes_total': 322122547200}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'epoch': 3, 'active_gid': 4100, 'active_name': 'x', 'active_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6800', 'nonce': 9734}, {'type': 'v1', 'addr': '172.21.15.160:6801', 'nonce': 9734}]}, 'active_addr': '172.21.15.160:6801/9734', 'active_change': '2020-05-24 13:49:45.331777', 'available': True, 'standbys': [], 'modules': ['iostat', 'restful'], 'available_modules': [{'name': 'alerts', 'can_run': True, 'error_string': '', 'module_options': {'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'How frequently to reexamine health status', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_destination': {'name': 'smtp_destination', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email address to send alerts to', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_from_name': {'name': 'smtp_from_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'Ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Email From: name', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_host': {'name': 'smtp_host', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_password': {'name': 'smtp_password', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Password to authenticate with', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_port': {'name': 'smtp_port', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '465', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP port', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_sender': {'name': 'smtp_sender', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'SMTP envelope sender', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_ssl': {'name': 'smtp_ssl', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Use SSL to connect to SMTP server', 'long_desc': '', 'tags': [], 'see_also': []}, 'smtp_user': {'name': 'smtp_user', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'User to authenticate as', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ansible', 'can_run': True, 'error_string': '', 'module_options': {'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_url': {'name': 'server_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_server': {'name': 'verify_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'balancer', 'can_run': True, 'error_string': '', 'module_options': {'active': {'name': 'active', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically balance PGs across cluster', 'long_desc': '', 'tags': [], 'see_also': []}, 'begin_time': {'name': 'begin_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '0000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'beginning time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'begin_weekday': {'name': 'begin_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to this day of the week or later', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'crush_compat_max_iterations': {'name': 'crush_compat_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '25', 'min': '1', 'max': '250', 'enum_allowed': [], 'desc': 'maximum number of iterations to attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'crush_compat_metrics': {'name': 'crush_compat_metrics', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'pgs,objects,bytes', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'metrics with which to calculate OSD utilization', 'long_desc': 'Value is a list of one or more of "pgs", "objects", or "bytes", and indicates which metrics to use to balance utilization.', 'tags': [], 'see_also': []}, 'crush_compat_step': {'name': 'crush_compat_step', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0.5', 'min': '0.001', 'max': '0.999', 'enum_allowed': [], 'desc': 'aggressiveness of optimization', 'long_desc': '.99 is very aggressive, .01 is less aggressive', 'tags': [], 'see_also': []}, 'end_time': {'name': 'end_time', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '2400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'ending time of day to automatically balance', 'long_desc': 'This is a time of day in the format HHMM.', 'tags': [], 'see_also': []}, 'end_weekday': {'name': 'end_weekday', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '7', 'min': '0', 'max': '7', 'enum_allowed': [], 'desc': 'Restrict automatic balancing to days of the week earlier than this', 'long_desc': '0 or 7 = Sunday, 1 = Monday, etc.', 'tags': [], 'see_also': []}, 'min_score': {'name': 'min_score', 'type': 'float', 'level': 'advanced', 'flags': 1, 'default_value': '0', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'minimum score, below which no optimization is attempted', 'long_desc': '', 'tags': [], 'see_also': []}, 'mode': {'name': 'mode', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'none', 'min': '', 'max': '', 'enum_allowed': ['crush-compat', 'none', 'upmap'], 'desc': 'Balancer mode', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_ids': {'name': 'pool_ids', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'pools which the automatic balancing will be limited to', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and attempt optimization', 'long_desc': '', 'tags': [], 'see_also': []}, 'upmap_max_deviation': {'name': 'upmap_max_deviation', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '1', 'max': '', 'enum_allowed': [], 'desc': 'deviation below which no optimization is attempted', 'long_desc': 'If the number of PGs are within this count then no optimization is attempted', 'tags': [], 'see_also': []}, 'upmap_max_iterations': {'name': 'upmap_max_iterations', 'type': 'uint', 'level': 'advanced', 'flags': 1, 'default_value': '10', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'maximum upmap optimization iterations', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'crash', 'can_run': True, 'error_string': '', 'module_options': {'retain_interval': {'name': 'retain_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '31536000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain crashes before pruning them', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_recent_interval': {'name': 'warn_recent_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '1209600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'time interval in which to warn about recent crashes', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'dashboard', 'can_run': True, 'error_string': '', 'module_options': {'ALERTMANAGER_API_HOST': {'name': 'ALERTMANAGER_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_ENABLED': {'name': 'AUDIT_API_ENABLED', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'AUDIT_API_LOG_PAYLOAD': {'name': 'AUDIT_API_LOG_PAYLOAD', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ENABLE_BROWSABLE_API': {'name': 'ENABLE_BROWSABLE_API', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_cephfs': {'name': 'FEATURE_TOGGLE_cephfs', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_iscsi': {'name': 'FEATURE_TOGGLE_iscsi', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_mirroring': {'name': 'FEATURE_TOGGLE_mirroring', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rbd': {'name': 'FEATURE_TOGGLE_rbd', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'FEATURE_TOGGLE_rgw': {'name': 'FEATURE_TOGGLE_rgw', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE': {'name': 'GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_PASSWORD': {'name': 'GRAFANA_API_PASSWORD', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_SSL_VERIFY': {'name': 'GRAFANA_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_URL': {'name': 'GRAFANA_API_URL', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_API_USERNAME': {'name': 'GRAFANA_API_USERNAME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'GRAFANA_UPDATE_DASHBOARDS': {'name': 'GRAFANA_UPDATE_DASHBOARDS', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ISCSI_API_SSL_VERIFICATION': {'name': 'ISCSI_API_SSL_VERIFICATION', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'PROMETHEUS_API_HOST': {'name': 'PROMETHEUS_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'REST_REQUESTS_TIMEOUT': {'name': 'REST_REQUESTS_TIMEOUT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '45', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ACCESS_KEY': {'name': 'RGW_API_ACCESS_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_ADMIN_RESOURCE': {'name': 'RGW_API_ADMIN_RESOURCE', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'admin', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_HOST': {'name': 'RGW_API_HOST', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_PORT': {'name': 'RGW_API_PORT', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '80', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SCHEME': {'name': 'RGW_API_SCHEME', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'http', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SECRET_KEY': {'name': 'RGW_API_SECRET_KEY', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_SSL_VERIFY': {'name': 'RGW_API_SSL_VERIFY', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'RGW_API_USER_ID': {'name': 'RGW_API_USER_ID', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'crt_file': {'name': 'crt_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'debug': {'name': 'debug', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Enable/disable debug options', 'long_desc': '', 'tags': [], 'see_also': []}, 'jwt_token_ttl': {'name': 'jwt_token_ttl', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '28800', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '::', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8080', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl_server_port': {'name': 'ssl_server_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '8443', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_behaviour': {'name': 'standby_behaviour', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'redirect', 'min': '', 'max': '', 'enum_allowed': ['error', 'redirect'], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'standby_error_status_code': {'name': 'standby_error_status_code', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '500', 'min': '400', 'max': '599', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url_prefix': {'name': 'url_prefix', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'deepsea', 'can_run': True, 'error_string': '', 'module_options': {'salt_api_eauth': {'name': 'salt_api_eauth', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'sharedsecret', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_password': {'name': 'salt_api_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_url': {'name': 'salt_api_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'salt_api_username': {'name': 'salt_api_username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'devicehealth', 'can_run': True, 'error_string': '', 'module_options': {'enable_monitoring': {'name': 'enable_monitoring', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'monitor device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'mark_out_threshold': {'name': 'mark_out_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '2419200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'automatically mark OSD if it may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}, 'pool_name': {'name': 'pool_name', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'device_health_metrics', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name of pool in which to store device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'retention_period': {'name': 'retention_period', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '15552000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how long to retain device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_frequency': {'name': 'scrape_frequency', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to scrape device health metrics', 'long_desc': '', 'tags': [], 'see_also': []}, 'self_heal': {'name': 'self_heal', 'type': 'bool', 'level': 'advanced', 'flags': 1, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'preemptively heal cluster around devices that may fail', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to wake up and check device health', 'long_desc': '', 'tags': [], 'see_also': []}, 'warn_threshold': {'name': 'warn_threshold', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '7257600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'raise health warning if OSD may fail before this long', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_cloud', 'can_run': True, 'error_string': '', 'module_options': {'diskprediction_cert_context': {'name': 'diskprediction_cert_context', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_default_authority': {'name': 'diskprediction_default_authority', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_password': {'name': 'diskprediction_password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_port': {'name': 'diskprediction_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '31400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_retrieve_prediction_interval': {'name': 'diskprediction_retrieve_prediction_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_server': {'name': 'diskprediction_server', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_ssl_target_name_override': {'name': 'diskprediction_ssl_target_name_override', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'localhost', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_metrics_interval': {'name': 'diskprediction_upload_metrics_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_upload_smart_interval': {'name': 'diskprediction_upload_smart_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '43200', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'diskprediction_user': {'name': 'diskprediction_user', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'diskprediction_local', 'can_run': True, 'error_string': '', 'module_options': {'predict_interval': {'name': 'predict_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '86400', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '600', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'influx', 'can_run': False, 'error_string': 'influxdb python module not found', 'module_options': {'batch_size': {'name': 'batch_size', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5000', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'database': {'name': 'database', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'ceph', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'hostname': {'name': 'hostname', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '30', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'password': {'name': 'password', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'port': {'name': 'port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '8086', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssl': {'name': 'ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'false', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'threads': {'name': 'threads', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'username': {'name': 'username', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'verify_ssl': {'name': 'verify_ssl', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'true', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'insights', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'iostat', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'localpool', 'can_run': True, 'error_string': '', 'module_options': {'failure_domain': {'name': 'failure_domain', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'host', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'failure domain for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'min_size': {'name': 'min_size', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default min_size for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'num_rep': {'name': 'num_rep', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '3', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default replica count for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'pg_num': {'name': 'pg_num', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '128', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'default pg_num for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'prefix': {'name': 'prefix', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'name prefix for any created local pool', 'long_desc': '', 'tags': [], 'see_also': []}, 'subtree': {'name': 'subtree', 'type': 'str', 'level': 'advanced', 'flags': 1, 'default_value': 'rack', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'CRUSH level for which to create a local pool', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'orchestrator_cli', 'can_run': True, 'error_string': '', 'module_options': {'orchestrator': {'name': 'orchestrator', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'pg_autoscaler', 'can_run': True, 'error_string': '', 'module_options': {'sleep_interval': {'name': 'sleep_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'progress', 'can_run': True, 'error_string': '', 'module_options': {'max_completed_events': {'name': 'max_completed_events', 'type': 'int', 'level': 'advanced', 'flags': 1, 'default_value': '50', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'number of past completed events to remember', 'long_desc': '', 'tags': [], 'see_also': []}, 'persist_interval': {'name': 'persist_interval', 'type': 'secs', 'level': 'advanced', 'flags': 1, 'default_value': '5', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'how frequently to persist completed events', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'prometheus', 'can_run': True, 'error_string': '', 'module_options': {'rbd_stats_pools': {'name': 'rbd_stats_pools', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rbd_stats_pools_refresh_interval': {'name': 'rbd_stats_pools_refresh_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'scrape_interval': {'name': 'scrape_interval', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rbd_support', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'restful', 'can_run': True, 'error_string': '', 'module_options': {'key_file': {'name': 'key_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_addr': {'name': 'server_addr', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'server_port': {'name': 'server_port', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'rook', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'selftest', 'can_run': True, 'error_string': '', 'module_options': {'roption1': {'name': 'roption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'roption2': {'name': 'roption2', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'xyz', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption1': {'name': 'rwoption1', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption2': {'name': 'rwoption2', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption3': {'name': 'rwoption3', 'type': 'float', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption4': {'name': 'rwoption4', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption5': {'name': 'rwoption5', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'rwoption6': {'name': 'rwoption6', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testkey': {'name': 'testkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testlkey': {'name': 'testlkey', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'testnewline': {'name': 'testnewline', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'ssh', 'can_run': True, 'error_string': '', 'module_options': {'inventory_cache_timeout_min': {'name': 'inventory_cache_timeout_min', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'ssh_config_file': {'name': 'ssh_config_file', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'status', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'telegraf', 'can_run': True, 'error_string': '', 'module_options': {'address': {'name': 'address', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'unixgram:///tmp/telegraf.sock', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '15', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'telemetry', 'can_run': True, 'error_string': '', 'module_options': {'channel_basic': {'name': 'channel_basic', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': 'Share basic cluster information (size, version)', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_crash': {'name': 'channel_crash', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_device': {'name': 'channel_device', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'True', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'channel_ident': {'name': 'channel_ident', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'contact': {'name': 'contact', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'description': {'name': 'description', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'device_url': {'name': 'device_url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/device', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'enabled': {'name': 'enabled', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '24', 'min': '8', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'last_opt_revision': {'name': 'last_opt_revision', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '1', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'leaderboard': {'name': 'leaderboard', 'type': 'bool', 'level': 'advanced', 'flags': 0, 'default_value': 'False', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'organization': {'name': 'organization', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'proxy': {'name': 'proxy', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'url': {'name': 'url', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'https://telemetry.ceph.com/report', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}, {'name': 'test_orchestrator', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'volumes', 'can_run': True, 'error_string': '', 'module_options': {}}, {'name': 'zabbix', 'can_run': True, 'error_string': '', 'module_options': {'identifier': {'name': 'identifier', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'interval': {'name': 'interval', 'type': 'secs', 'level': 'advanced', 'flags': 0, 'default_value': '60', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_host': {'name': 'zabbix_host', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': 'None', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_port': {'name': 'zabbix_port', 'type': 'int', 'level': 'advanced', 'flags': 0, 'default_value': '10051', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}, 'zabbix_sender': {'name': 'zabbix_sender', 'type': 'str', 'level': 'advanced', 'flags': 0, 'default_value': '/usr/bin/zabbix_sender', 'min': '', 'max': '', 'enum_allowed': [], 'desc': '', 'long_desc': '', 'tags': [], 'see_also': []}}}], 'services': {}, 'always_on_modules': {'nautilus': ['balancer', 'crash', 'devicehealth', 'orchestrator_cli', 'progress', 'rbd_support', 'status', 'volumes']}}, 'servicemap': {'epoch': 1, 'modified': '0.000000', 'services': {}}, 'progress_events': {}} 2020-05-24T13:50:21.186 INFO:tasks.ceph.ceph_manager.c2:clean! 2020-05-24T13:50:21.187 INFO:tasks.ceph:Waiting until ceph cluster c2 is healthy... 2020-05-24T13:50:21.187 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 health 2020-05-24T13:50:22.163 INFO:teuthology.misc.health.smithi160.stdout:HEALTH_OK 2020-05-24T13:50:22.185 DEBUG:teuthology.misc:Ceph health: HEALTH_OK 2020-05-24T13:50:22.186 INFO:teuthology.run_tasks:Running task rgw... 2020-05-24T13:50:22.199 DEBUG:tasks.rgw:config is {'c1.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c1.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.0': {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'c2.client.1': {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2020-05-24T13:50:22.200 DEBUG:tasks.rgw:client list is dict_keys(['c1.client.0', 'c1.client.1', 'c2.client.0', 'c2.client.1']) 2020-05-24T13:50:22.200 INFO:tasks.rgw:Creating data pools 2020-05-24T13:50:22.200 DEBUG:tasks.rgw:Obtaining remote for client c1.client.0 2020-05-24T13:50:22.201 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2020-05-24T13:50:24.103 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'default.rgw.buckets.data' created 2020-05-24T13:50:24.125 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2020-05-24T13:50:25.103 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2020-05-24T13:50:25.116 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2020-05-24T13:50:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'default.rgw.buckets.index' created 2020-05-24T13:50:26.179 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2020-05-24T13:50:27.281 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2020-05-24T13:50:27.300 DEBUG:tasks.rgw:Obtaining remote for client c1.client.1 2020-05-24T13:50:27.300 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2020-05-24T13:50:28.062 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'default.rgw.buckets.data' already exists 2020-05-24T13:50:28.075 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2020-05-24T13:50:29.436 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2020-05-24T13:50:29.452 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2020-05-24T13:50:30.203 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'default.rgw.buckets.index' already exists 2020-05-24T13:50:30.217 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2020-05-24T13:50:31.522 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2020-05-24T13:50:31.534 DEBUG:tasks.rgw:Obtaining remote for client c2.client.0 2020-05-24T13:50:31.535 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2020-05-24T13:50:32.983 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'default.rgw.buckets.data' created 2020-05-24T13:50:33.005 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2020-05-24T13:50:33.962 INFO:teuthology.orchestra.run.smithi160.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2020-05-24T13:50:33.980 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2020-05-24T13:50:35.113 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'default.rgw.buckets.index' created 2020-05-24T13:50:35.126 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2020-05-24T13:50:36.089 INFO:teuthology.orchestra.run.smithi160.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2020-05-24T13:50:36.103 DEBUG:tasks.rgw:Obtaining remote for client c2.client.1 2020-05-24T13:50:36.103 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2020-05-24T13:50:36.871 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'default.rgw.buckets.data' already exists 2020-05-24T13:50:36.879 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2020-05-24T13:50:38.240 INFO:teuthology.orchestra.run.smithi160.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2020-05-24T13:50:38.254 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2020-05-24T13:50:38.936 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'default.rgw.buckets.index' already exists 2020-05-24T13:50:38.948 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2020-05-24T13:50:40.597 INFO:teuthology.orchestra.run.smithi160.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2020-05-24T13:50:40.610 DEBUG:tasks.rgw:Pools created 2020-05-24T13:50:40.611 INFO:tasks.rgw:Configuring compression type = random 2020-05-24T13:50:40.611 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'list'] 2020-05-24T13:50:40.612 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'list'] 2020-05-24T13:50:40.612 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user list 2020-05-24T13:50:40.876 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:40.879 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:44.212 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:44.215 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:44.213 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:44.215 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:44.359 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:44.361 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:44.359 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:44.361 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:44.890 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:44.892 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:44.890 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:44.892 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:44.918 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:44.921 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:46.568 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.570 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:46.607 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.610 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:46.608 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.610 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:46.628 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.631 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:46.628 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.631 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:46.701 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.703 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:46.701 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.703 7f760801c840 10 cannot find current period zonegroup using local zonegroup 2020-05-24T13:50:46.701 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.703 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:46.714 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.717 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:46.715 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.717 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:46.736 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.738 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:46.736 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.738 7f760801c840 10 Creating default zonegroup 2020-05-24T13:50:46.736 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.738 7f760801c840 20 get_system_obj_state: rctx=0x7ffc9e166ab0 obj=:.pools.avail state=0x563ad18b6a60 s->prefetch_data=0 2020-05-24T13:50:46.738 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:46.741 7f760801c840 10 couldn't find old data placement pools config, setting up new ones for the zone 2020-05-24T13:50:47.347 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.350 7f760801c840 10 WARNING: store->list_zones() returned r=-2 2020-05-24T13:50:47.348 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.350 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:47.363 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.366 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:47.951 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.954 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:47.964 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.966 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:47.964 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.966 7f760801c840 10 could not read realm id: (2) No such file or directory 2020-05-24T13:50:47.964 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.966 7f760801c840 10 WARNING: failed to set zone as default, r=-22 2020-05-24T13:50:47.964 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.966 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:47.980 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:47.982 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:48.462 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.465 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:48.518 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.518 7f760801c840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:50:48.519 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.518 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:48.592 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.595 7f760801c840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:50:48.592 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.595 7f760801c840 20 zonegroup default 2020-05-24T13:50:48.593 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.595 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:48.608 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.611 7f760801c840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:50:48.608 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.611 7f760801c840 10 Cannot find current period zone using local zone 2020-05-24T13:50:48.608 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.611 7f760801c840 10 Using default name default 2020-05-24T13:50:48.609 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.611 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:48.634 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.637 7f760801c840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:50:48.635 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.637 7f760801c840 20 rados->read ofs=0 len=0 2020-05-24T13:50:48.661 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.663 7f760801c840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:50:48.661 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:48.663 7f760801c840 20 zone default 2020-05-24T13:50:53.470 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.472 7f760801c840 20 add_watcher() i=0 2020-05-24T13:50:53.471 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.472 7f760801c840 20 add_watcher() i=1 2020-05-24T13:50:53.471 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.472 7f760801c840 20 add_watcher() i=2 2020-05-24T13:50:53.471 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.472 7f760801c840 20 add_watcher() i=3 2020-05-24T13:50:53.471 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.472 7f760801c840 20 add_watcher() i=4 2020-05-24T13:50:53.471 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.472 7f760801c840 20 add_watcher() i=5 2020-05-24T13:50:53.472 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.472 7f760801c840 20 add_watcher() i=6 2020-05-24T13:50:53.546 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.549 7f760801c840 20 add_watcher() i=7 2020-05-24T13:50:53.546 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.549 7f760801c840 2 all 8 watchers are set, enabling cache 2020-05-24T13:50:53.582 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:53.585 7f75ccff9700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:50:55.127 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:55.130 7f760801c840 10 rgw_init_ioctx warning: failed to set pg_autoscale_bias on default.rgw.meta 2020-05-24T13:50:55.130 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:55.133 7f760801c840 10 rgw_init_ioctx warning: failed to set pg_num_min on default.rgw.meta 2020-05-24T13:50:57.143 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:57.145 7f760801c840 10 rgw_init_ioctx warning: failed to set pg_autoscale_bias on default.rgw.log 2020-05-24T13:50:57.144 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:57.147 7f760801c840 10 rgw_init_ioctx warning: failed to set pg_num_min on default.rgw.log 2020-05-24T13:50:57.146 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:57.148 7f75aafe5700 20 reqs_thread_entry: start 2020-05-24T13:50:57.146 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:57.148 7f75a8fe1700 20 reqs_thread_entry: start 2020-05-24T13:50:57.146 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:57.149 7f760801c840 20 init_complete bucket index max shards: 0 2020-05-24T13:50:57.146 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:50:57.149 7f7536ffd700 20 reqs_thread_entry: start 2020-05-24T13:51:02.059 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.061 7f760801c840 20 remove_watcher() i=0 2020-05-24T13:51:02.059 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.061 7f760801c840 2 removed watcher, disabling cache 2020-05-24T13:51:02.154 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.156 7f760801c840 20 remove_watcher() i=1 2020-05-24T13:51:02.223 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.226 7f760801c840 20 remove_watcher() i=2 2020-05-24T13:51:02.284 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.287 7f760801c840 20 remove_watcher() i=3 2020-05-24T13:51:02.370 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.373 7f760801c840 20 remove_watcher() i=4 2020-05-24T13:51:02.454 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.457 7f760801c840 20 remove_watcher() i=5 2020-05-24T13:51:02.521 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.524 7f760801c840 20 remove_watcher() i=6 2020-05-24T13:51:02.603 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.606 7f760801c840 20 remove_watcher() i=7 2020-05-24T13:51:02.611 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2020-05-24T13:51:02.611 DEBUG:tasks.util.rgw: json result: [] 2020-05-24T13:51:02.612 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:02.613 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:02.614 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2020-05-24T13:51:02.918 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.920 7f812cff9700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:02.918 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.920 7f811ffff700 20 reqs_thread_entry: start 2020-05-24T13:51:02.919 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.920 7f8163ec3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:02.954 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.956 7f8163ec3840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:02.954 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.956 7f8163ec3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:02.971 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.974 7f8163ec3840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:02.971 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.974 7f8163ec3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.989 7f8163ec3840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:02.989 7f8163ec3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.007 7f8163ec3840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.007 7f8163ec3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.047 7f8163ec3840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:03.221 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"ed6137bd-5618-40aa-bc3d-51f3dd2cb043","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:51:03.221 DEBUG:tasks.util.rgw: json result: {'id': 'ed6137bd-5618-40aa-bc3d-51f3dd2cb043', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'metadata_heap': '', 'realm_id': ''} 2020-05-24T13:51:03.222 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['user', 'list'] 2020-05-24T13:51:03.222 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'user', 'list'] 2020-05-24T13:51:03.223 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 user list 2020-05-24T13:51:03.482 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.485 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:03.530 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.533 7ffbf7b80840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:03.530 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.533 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:03.577 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.580 7ffbf7b80840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:03.578 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.580 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.601 7ffbf7b80840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.601 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:03.625 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:03.628 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:04.530 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.533 7ffbf7b80840 20 RGWRados::pool_iterate: got zonegroup_info.e91ff828-547a-430c-9268-605ea7077fbe 2020-05-24T13:51:04.530 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.533 7ffbf7b80840 20 RGWRados::pool_iterate: got zone_info.ed6137bd-5618-40aa-bc3d-51f3dd2cb043 2020-05-24T13:51:04.531 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.533 7ffbf7b80840 20 RGWRados::pool_iterate: got zone_names.default 2020-05-24T13:51:04.531 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.533 7ffbf7b80840 20 RGWRados::pool_iterate: got zonegroups_names.default 2020-05-24T13:51:04.531 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.533 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.544 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.548 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:04.545 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.548 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.563 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.567 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:04.564 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.567 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.578 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.582 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:04.578 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.582 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.594 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.597 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:04.594 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.597 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.611 7ffbf7b80840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.611 7ffbf7b80840 10 cannot find current period zonegroup using local zonegroup 2020-05-24T13:51:04.608 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.611 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.626 7ffbf7b80840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.626 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.646 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.646 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.664 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.668 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:04.665 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.668 7ffbf7b80840 20 zonegroup default 2020-05-24T13:51:04.665 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.668 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.683 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.686 7ffbf7b80840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:04.683 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.686 7ffbf7b80840 10 Cannot find current period zone using local zone 2020-05-24T13:51:04.684 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.686 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.695 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.699 7ffbf7b80840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:04.696 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.699 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.710 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.713 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:04.711 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.714 7ffbf7b80840 20 rados->read ofs=0 len=0 2020-05-24T13:51:04.733 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.736 7ffbf7b80840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:04.734 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:04.736 7ffbf7b80840 20 zone default 2020-05-24T13:51:05.483 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.485 7ffbf7b80840 20 add_watcher() i=0 2020-05-24T13:51:05.483 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.485 7ffbf7b80840 20 add_watcher() i=1 2020-05-24T13:51:05.483 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.485 7ffbf7b80840 20 add_watcher() i=2 2020-05-24T13:51:05.484 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.485 7ffbf7b80840 20 add_watcher() i=3 2020-05-24T13:51:05.484 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.485 7ffbf7b80840 20 add_watcher() i=4 2020-05-24T13:51:05.484 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.485 7ffbf7b80840 20 add_watcher() i=5 2020-05-24T13:51:05.484 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.485 7ffbf7b80840 20 add_watcher() i=6 2020-05-24T13:51:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.533 7ffbf7b80840 20 add_watcher() i=7 2020-05-24T13:51:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.533 7ffbf7b80840 2 all 8 watchers are set, enabling cache 2020-05-24T13:51:05.559 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.562 7ffbc0ff9700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.572 7ffba37de700 20 reqs_thread_entry: start 2020-05-24T13:51:05.569 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.572 7ffba17da700 20 reqs_thread_entry: start 2020-05-24T13:51:05.570 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.572 7ffbf7b80840 20 init_complete bucket index max shards: 0 2020-05-24T13:51:05.570 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:05.573 7ffb337fe700 20 reqs_thread_entry: start 2020-05-24T13:51:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.341 7ffbf7b80840 20 remove_watcher() i=0 2020-05-24T13:51:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.341 7ffbf7b80840 2 removed watcher, disabling cache 2020-05-24T13:51:06.401 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.404 7ffbf7b80840 20 remove_watcher() i=1 2020-05-24T13:51:06.460 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.463 7ffbf7b80840 20 remove_watcher() i=2 2020-05-24T13:51:06.537 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.540 7ffbf7b80840 20 remove_watcher() i=3 2020-05-24T13:51:06.610 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.613 7ffbf7b80840 20 remove_watcher() i=4 2020-05-24T13:51:06.668 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.671 7ffbf7b80840 20 remove_watcher() i=5 2020-05-24T13:51:06.741 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.744 7ffbf7b80840 20 remove_watcher() i=6 2020-05-24T13:51:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:06.834 7ffbf7b80840 20 remove_watcher() i=7 2020-05-24T13:51:06.836 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2020-05-24T13:51:06.837 DEBUG:tasks.util.rgw: json result: [] 2020-05-24T13:51:06.837 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:06.837 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:06.838 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2020-05-24T13:51:07.062 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.064 7fc165ffb700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:07.062 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.064 7fc1657fa700 20 reqs_thread_entry: start 2020-05-24T13:51:07.062 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.065 7fc1a0e4f840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.092 7fc1a0e4f840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:07.089 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.092 7fc1a0e4f840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.103 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.106 7fc1a0e4f840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:07.104 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.107 7fc1a0e4f840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.117 7fc1a0e4f840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:07.114 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.117 7fc1a0e4f840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.126 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.130 7fc1a0e4f840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:07.127 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.130 7fc1a0e4f840 20 rados->read ofs=0 len=0 2020-05-24T13:51:07.155 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:07.158 7fc1a0e4f840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:07.254 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"ed6137bd-5618-40aa-bc3d-51f3dd2cb043","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:51:07.254 DEBUG:tasks.util.rgw: json result: {'id': 'ed6137bd-5618-40aa-bc3d-51f3dd2cb043', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'metadata_heap': '', 'realm_id': ''} 2020-05-24T13:51:07.255 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['user', 'list'] 2020-05-24T13:51:07.255 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'user', 'list'] 2020-05-24T13:51:07.256 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 user list 2020-05-24T13:51:07.524 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:07.524 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:09.637 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:09.637 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:09.637 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:10.258 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:10.258 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:10.258 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:10.258 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:10.329 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:10.330 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:10.330 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:10.330 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:10.373 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:10.373 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:11.729 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.729 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:11.738 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.739 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:11.739 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.739 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:11.751 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.752 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:11.752 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.752 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:11.793 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.793 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:11.793 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.793 7f935d867840 10 cannot find current period zonegroup using local zonegroup 2020-05-24T13:51:11.794 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.793 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:11.803 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.804 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:11.803 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.804 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:11.814 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.815 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:11.814 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.815 7f935d867840 10 Creating default zonegroup 2020-05-24T13:51:11.815 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.815 7f935d867840 20 get_system_obj_state: rctx=0x7ffca8e51730 obj=:.pools.avail state=0x56265df7f600 s->prefetch_data=0 2020-05-24T13:51:11.819 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:11.819 7f935d867840 10 couldn't find old data placement pools config, setting up new ones for the zone 2020-05-24T13:51:12.282 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:12.282 7f935d867840 10 WARNING: store->list_zones() returned r=-2 2020-05-24T13:51:12.282 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:12.282 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:12.292 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:12.292 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:13.128 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.128 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:13.155 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.155 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:13.155 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.155 7f935d867840 10 could not read realm id: (2) No such file or directory 2020-05-24T13:51:13.155 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.155 7f935d867840 10 WARNING: failed to set zone as default, r=-22 2020-05-24T13:51:13.155 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.155 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:13.169 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.169 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:13.424 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.424 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:13.484 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.485 7f935d867840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:13.485 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.485 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:13.511 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.512 7f935d867840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:13.512 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.512 7f935d867840 20 zonegroup default 2020-05-24T13:51:13.512 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.512 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:13.534 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.534 7f935d867840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:13.534 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.534 7f935d867840 10 Cannot find current period zone using local zone 2020-05-24T13:51:13.534 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.534 7f935d867840 10 Using default name default 2020-05-24T13:51:13.534 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.534 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:13.598 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.599 7f935d867840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:13.599 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.599 7f935d867840 20 rados->read ofs=0 len=0 2020-05-24T13:51:13.626 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.626 7f935d867840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:13.626 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:13.626 7f935d867840 20 zone default 2020-05-24T13:51:18.742 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.742 7f935d867840 20 add_watcher() i=0 2020-05-24T13:51:18.742 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.742 7f935d867840 20 add_watcher() i=1 2020-05-24T13:51:18.742 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.742 7f935d867840 20 add_watcher() i=2 2020-05-24T13:51:18.743 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.742 7f935d867840 20 add_watcher() i=3 2020-05-24T13:51:18.743 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.742 7f935d867840 20 add_watcher() i=4 2020-05-24T13:51:18.743 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.742 7f935d867840 20 add_watcher() i=5 2020-05-24T13:51:18.763 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.763 7f935d867840 20 add_watcher() i=6 2020-05-24T13:51:18.978 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.979 7f935d867840 20 add_watcher() i=7 2020-05-24T13:51:18.979 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:18.979 7f935d867840 2 all 8 watchers are set, enabling cache 2020-05-24T13:51:19.012 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:19.013 7f931affd700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:20.523 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:20.523 7f935d867840 10 rgw_init_ioctx warning: failed to set pg_autoscale_bias on default.rgw.meta 2020-05-24T13:51:20.528 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:20.529 7f935d867840 10 rgw_init_ioctx warning: failed to set pg_num_min on default.rgw.meta 2020-05-24T13:51:22.601 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:22.601 7f935d867840 10 rgw_init_ioctx warning: failed to set pg_autoscale_bias on default.rgw.log 2020-05-24T13:51:22.603 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:22.604 7f935d867840 10 rgw_init_ioctx warning: failed to set pg_num_min on default.rgw.log 2020-05-24T13:51:22.604 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:22.605 7f93017e2700 20 reqs_thread_entry: start 2020-05-24T13:51:22.605 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:22.605 7f92c6ffd700 20 reqs_thread_entry: start 2020-05-24T13:51:22.605 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:22.605 7f935d867840 20 init_complete bucket index max shards: 0 2020-05-24T13:51:22.605 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:22.605 7f92c57fa700 20 reqs_thread_entry: start 2020-05-24T13:51:27.628 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:27.628 7f935d867840 20 remove_watcher() i=0 2020-05-24T13:51:27.628 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:27.628 7f935d867840 2 removed watcher, disabling cache 2020-05-24T13:51:27.700 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:27.700 7f935d867840 20 remove_watcher() i=1 2020-05-24T13:51:27.785 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:27.786 7f935d867840 20 remove_watcher() i=2 2020-05-24T13:51:27.851 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:27.852 7f935d867840 20 remove_watcher() i=3 2020-05-24T13:51:27.897 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:27.897 7f935d867840 20 remove_watcher() i=4 2020-05-24T13:51:27.952 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:27.953 7f935d867840 20 remove_watcher() i=5 2020-05-24T13:51:28.013 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.014 7f935d867840 20 remove_watcher() i=6 2020-05-24T13:51:28.083 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.083 7f935d867840 20 remove_watcher() i=7 2020-05-24T13:51:28.088 INFO:teuthology.orchestra.run.smithi160.stdout:[] 2020-05-24T13:51:28.088 DEBUG:tasks.util.rgw: json result: [] 2020-05-24T13:51:28.089 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:28.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:28.089 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2020-05-24T13:51:28.285 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.285 7f2203fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:28.285 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.285 7f22037fe700 20 reqs_thread_entry: start 2020-05-24T13:51:28.286 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.285 7f223eb5b840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.312 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.312 7f223eb5b840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:28.312 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.312 7f223eb5b840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.343 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.343 7f223eb5b840 20 rados_obj.operate() r=0 bl.length=805 2020-05-24T13:51:28.343 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.343 7f223eb5b840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.356 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.357 7f223eb5b840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:28.357 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.357 7f223eb5b840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.372 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.372 7f223eb5b840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:28.372 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.372 7f223eb5b840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.393 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.393 7f223eb5b840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:28.601 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"f912e7d4-bf23-4231-8fa3-e725aba63ce0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:51:28.601 DEBUG:tasks.util.rgw: json result: {'id': 'f912e7d4-bf23-4231-8fa3-e725aba63ce0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'metadata_heap': '', 'realm_id': ''} 2020-05-24T13:51:28.602 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['user', 'list'] 2020-05-24T13:51:28.602 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'user', 'list'] 2020-05-24T13:51:28.602 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 user list 2020-05-24T13:51:28.819 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.820 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.868 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.869 7fa7b0fb6840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:28.869 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.869 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.898 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.898 7fa7b0fb6840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:28.898 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.898 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.913 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.913 7fa7b0fb6840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:28.913 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.913 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:28.932 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:28.932 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:29.521 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.521 7fa7b0fb6840 20 RGWRados::pool_iterate: got zonegroup_info.4fc0eca6-144b-4150-ad12-a645949f73c8 2020-05-24T13:51:29.521 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.521 7fa7b0fb6840 20 RGWRados::pool_iterate: got zone_info.f912e7d4-bf23-4231-8fa3-e725aba63ce0 2020-05-24T13:51:29.521 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.521 7fa7b0fb6840 20 RGWRados::pool_iterate: got zone_names.default 2020-05-24T13:51:29.522 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.521 7fa7b0fb6840 20 RGWRados::pool_iterate: got zonegroups_names.default 2020-05-24T13:51:29.522 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.521 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.538 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.538 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:29.538 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.538 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.553 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.554 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:29.554 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.554 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.571 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.572 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:29.572 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.572 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.586 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.587 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:29.587 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.587 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.602 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.603 7fa7b0fb6840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:29.602 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.603 7fa7b0fb6840 10 cannot find current period zonegroup using local zonegroup 2020-05-24T13:51:29.603 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.603 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.614 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.615 7fa7b0fb6840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:29.615 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.615 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.632 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.633 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:29.633 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.633 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.651 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.652 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:29.652 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.652 7fa7b0fb6840 20 zonegroup default 2020-05-24T13:51:29.652 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.652 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.666 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.666 7fa7b0fb6840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:29.666 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.666 7fa7b0fb6840 10 Cannot find current period zone using local zone 2020-05-24T13:51:29.666 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.666 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.679 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.680 7fa7b0fb6840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:29.679 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.680 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.696 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.697 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:29.696 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.697 7fa7b0fb6840 20 rados->read ofs=0 len=0 2020-05-24T13:51:29.714 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.714 7fa7b0fb6840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:29.714 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:29.714 7fa7b0fb6840 20 zone default 2020-05-24T13:51:30.320 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.320 7fa7b0fb6840 20 add_watcher() i=0 2020-05-24T13:51:30.320 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.320 7fa7b0fb6840 20 add_watcher() i=1 2020-05-24T13:51:30.320 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.320 7fa7b0fb6840 20 add_watcher() i=2 2020-05-24T13:51:30.321 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.320 7fa7b0fb6840 20 add_watcher() i=3 2020-05-24T13:51:30.321 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.320 7fa7b0fb6840 20 add_watcher() i=4 2020-05-24T13:51:30.321 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.320 7fa7b0fb6840 20 add_watcher() i=5 2020-05-24T13:51:30.321 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.320 7fa7b0fb6840 20 add_watcher() i=6 2020-05-24T13:51:30.371 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.371 7fa7b0fb6840 20 add_watcher() i=7 2020-05-24T13:51:30.372 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.371 7fa7b0fb6840 2 all 8 watchers are set, enabling cache 2020-05-24T13:51:30.403 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.403 7fa775ffb700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:30.410 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.411 7fa73ffff700 20 reqs_thread_entry: start 2020-05-24T13:51:30.411 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.411 7fa73dffb700 20 reqs_thread_entry: start 2020-05-24T13:51:30.411 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.411 7fa7b0fb6840 20 init_complete bucket index max shards: 0 2020-05-24T13:51:30.411 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:30.412 7fa6fffff700 20 reqs_thread_entry: start 2020-05-24T13:51:31.215 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.215 7fa7b0fb6840 20 remove_watcher() i=0 2020-05-24T13:51:31.215 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.215 7fa7b0fb6840 2 removed watcher, disabling cache 2020-05-24T13:51:31.321 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.322 7fa7b0fb6840 20 remove_watcher() i=1 2020-05-24T13:51:31.396 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.393 7fa7b0fb6840 20 remove_watcher() i=2 2020-05-24T13:51:31.450 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.450 7fa7b0fb6840 20 remove_watcher() i=3 2020-05-24T13:51:31.505 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.505 7fa7b0fb6840 20 remove_watcher() i=4 2020-05-24T13:51:31.562 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.563 7fa7b0fb6840 20 remove_watcher() i=5 2020-05-24T13:51:31.617 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.613 7fa7b0fb6840 20 remove_watcher() i=6 2020-05-24T13:51:31.663 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.663 7fa7b0fb6840 20 remove_watcher() i=7 2020-05-24T13:51:31.667 INFO:teuthology.orchestra.run.smithi160.stdout:[] 2020-05-24T13:51:31.668 DEBUG:tasks.util.rgw: json result: [] 2020-05-24T13:51:31.668 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:31.669 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2020-05-24T13:51:31.669 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2020-05-24T13:51:31.962 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.962 7f3baa7fc700 2 RGWDataChangesLog::ChangesRenewThread: start 2020-05-24T13:51:31.962 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.962 7f3ba9ffb700 20 reqs_thread_entry: start 2020-05-24T13:51:31.962 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.962 7f3be51e3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:31.996 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.996 7f3be51e3840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:31.996 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:31.996 7f3be51e3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.022 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:32.023 7f3be51e3840 20 rados_obj.operate() r=0 bl.length=821 2020-05-24T13:51:32.023 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:32.023 7f3be51e3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.033 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:32.033 7f3be51e3840 20 rados_obj.operate() r=-2 bl.length=0 2020-05-24T13:51:32.033 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:32.033 7f3be51e3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.055 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:32.055 7f3be51e3840 20 rados_obj.operate() r=0 bl.length=46 2020-05-24T13:51:32.055 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:32.055 7f3be51e3840 20 rados->read ofs=0 len=0 2020-05-24T13:51:32.074 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:51:32.074 7f3be51e3840 20 rados_obj.operate() r=0 bl.length=348 2020-05-24T13:51:32.159 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"f912e7d4-bf23-4231-8fa3-e725aba63ce0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:51:32.159 DEBUG:tasks.util.rgw: json result: {'id': 'f912e7d4-bf23-4231-8fa3-e725aba63ce0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'metadata_heap': '', 'realm_id': ''} 2020-05-24T13:51:32.160 INFO:tasks.rgw:Starting rgw... 2020-05-24T13:51:32.160 INFO:tasks.rgw:rgw c1.client.0 config is {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2020-05-24T13:51:32.160 INFO:tasks.rgw:Using civetweb as radosgw frontend 2020-05-24T13:51:32.161 DEBUG:teuthology.misc:running c1.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2020-05-24T13:51:32.161 INFO:tasks.rgw.c1.client.0:Restarting daemon 2020-05-24T13:51:32.162 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2020-05-24T13:51:32.166 INFO:tasks.rgw.c1.client.0:Started 2020-05-24T13:51:32.166 INFO:tasks.rgw:rgw c1.client.1 config is {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2020-05-24T13:51:32.167 INFO:tasks.rgw:Using civetweb as radosgw frontend 2020-05-24T13:51:32.167 DEBUG:teuthology.misc:running c1.client.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c1.client.1.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2020-05-24T13:51:32.167 INFO:tasks.rgw.c1.client.1:Restarting daemon 2020-05-24T13:51:32.168 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2020-05-24T13:51:32.171 INFO:tasks.rgw.c1.client.1:Started 2020-05-24T13:51:32.171 INFO:tasks.rgw:rgw c2.client.0 config is {'port': 8000, 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2020-05-24T13:51:32.172 INFO:tasks.rgw:Using civetweb as radosgw frontend 2020-05-24T13:51:32.172 DEBUG:teuthology.misc:running c2.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2020-05-24T13:51:32.172 INFO:tasks.rgw.c2.client.0:Restarting daemon 2020-05-24T13:51:32.173 INFO:teuthology.orchestra.run.smithi160:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2020-05-24T13:51:32.175 INFO:tasks.rgw.c2.client.0:Started 2020-05-24T13:51:32.176 INFO:tasks.rgw:rgw c2.client.1 config is {'port': 8001, 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2020-05-24T13:51:32.176 INFO:tasks.rgw:Using civetweb as radosgw frontend 2020-05-24T13:51:32.176 DEBUG:teuthology.misc:running c2.client.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/c2.client.1.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2020-05-24T13:51:32.177 INFO:tasks.rgw.c2.client.1:Restarting daemon 2020-05-24T13:51:32.177 INFO:teuthology.orchestra.run.smithi160:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2020-05-24T13:51:32.180 INFO:tasks.rgw.c2.client.1:Started 2020-05-24T13:51:32.180 INFO:tasks.rgw:Polling c1.client.0 until it starts accepting connections on http://smithi050.front.sepia.ceph.com:8000/ 2020-05-24T13:51:32.181 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000/ 2020-05-24T13:51:32.280 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:32.281 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:32.298 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:51:32.299 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:51:33.300 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000/ 2020-05-24T13:51:33.384 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:33.387 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:33.387 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:51:33.388 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:51:35.388 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000/ 2020-05-24T13:51:35.449 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:35.450 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:35.453 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:51:35.454 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:51:39.455 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000/ 2020-05-24T13:51:39.522 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:39.523 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:39.526 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:51:39.527 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:51:47.528 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000/ 2020-05-24T13:51:47.573 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:47.574 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:47.806 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 919 0 --:--:-- --:--:-- --:--:-- 922 2020-05-24T13:51:47.807 INFO:teuthology.orchestra.run.smithi050.stdout:anonymous 2020-05-24T13:51:47.807 INFO:tasks.rgw:Polling c1.client.1 until it starts accepting connections on http://smithi050.front.sepia.ceph.com:8001/ 2020-05-24T13:51:47.808 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001/ 2020-05-24T13:51:47.856 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:47.857 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:48.117 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 822 0 --:--:-- --:--:-- --:--:-- 823 2020-05-24T13:51:48.117 INFO:teuthology.orchestra.run.smithi050.stdout:anonymous 2020-05-24T13:51:48.117 INFO:tasks.rgw:Polling c2.client.0 until it starts accepting connections on http://smithi160.front.sepia.ceph.com:8000/ 2020-05-24T13:51:48.118 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8000/ 2020-05-24T13:51:48.183 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:48.184 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:48.439 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 836 0 --:--:-- --:--:-- --:--:-- 839 2020-05-24T13:51:48.439 INFO:teuthology.orchestra.run.smithi160.stdout:anonymous 2020-05-24T13:51:48.440 INFO:tasks.rgw:Polling c2.client.1 until it starts accepting connections on http://smithi160.front.sepia.ceph.com:8001/ 2020-05-24T13:51:48.440 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8001/ 2020-05-24T13:51:48.488 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:51:48.489 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:51:48.753 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 809 0 --:--:-- --:--:-- --:--:-- 810 2020-05-24T13:51:48.753 INFO:teuthology.orchestra.run.smithi160.stdout:anonymous 2020-05-24T13:51:48.753 INFO:teuthology.run_tasks:Running task rgw-multisite... 2020-05-24T13:51:48.964 INFO:tasks.rgw_multisite:creating realm.. 2020-05-24T13:51:48.965 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:51:48.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:51:48.967 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 realm create --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:51:50.391 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","name":"test-realm","current_period":"07b6e218-7bbb-4b4a-8656-8ecffcf68225","epoch":1} 2020-05-24T13:51:50.392 INFO:tasks.rgw_multisite:creating master zonegroup.. 2020-05-24T13:51:50.392 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:51:50.392 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:51:50.393 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup set --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:51:50.770 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:51:50.774 7fc4b4e5b840 -1 old format 2020-05-24T13:51:51.315 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"","zones":[],"placement_targets":[],"default_placement":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T13:51:51.316 INFO:tasks.rgw_multisite:creating master zone.. 2020-05-24T13:51:51.317 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--endpoints', 'http://smithi050.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:51:51.317 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--endpoints', 'http://smithi050.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:51:51.317 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --master --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --endpoints http://smithi050.front.sepia.ceph.com:8000 --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:51:53.825 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T13:51:53.826 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool create test-zone1.rgw.buckets.data 64 64 --cluster c1 2020-05-24T13:51:54.731 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'test-zone1.rgw.buckets.data' created 2020-05-24T13:51:54.744 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool application enable test-zone1.rgw.buckets.data rgw --cluster c1 2020-05-24T13:51:55.809 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rgw' on pool 'test-zone1.rgw.buckets.data' 2020-05-24T13:51:55.821 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:51:55.822 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:51:55.822 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:52:01.620 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:52:01.621 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:01.621 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:01.622 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:52:27.720 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","epoch":1,"predecessor_uuid":"07b6e218-7bbb-4b4a-8656-8ecffcf68225","sync_status":[],"period_map":{"id":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"false","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":2} 2020-05-24T13:52:27.722 DEBUG:tasks.rgw.c1.client.0:waiting for process to exit 2020-05-24T13:52:27.722 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T13:52:27.740 INFO:tasks.rgw.c1.client.0.smithi050.stdout:2020-05-24 13:52:27.732 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends civetweb port=8000 -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground (PID: 13118) UID: 0 2020-05-24T13:52:27.741 INFO:tasks.rgw.c1.client.0.smithi050.stdout:2020-05-24 13:52:27.734 4049040 -1 shutting down 2020-05-24T13:52:39.930 INFO:tasks.rgw.c1.client.0:Stopped 2020-05-24T13:52:39.932 INFO:tasks.rgw.c1.client.0:Restarting daemon 2020-05-24T13:52:39.932 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2020-05-24T13:52:39.939 INFO:tasks.rgw.c1.client.0:Started 2020-05-24T13:52:39.939 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000 2020-05-24T13:52:40.038 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:40.039 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:40.045 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:52:40.046 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:52:41.047 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000 2020-05-24T13:52:41.139 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:41.142 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:41.142 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:52:41.143 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:52:43.145 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000 2020-05-24T13:52:43.234 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:43.236 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:43.240 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:52:43.241 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:52:47.242 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000 2020-05-24T13:52:47.329 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:47.329 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:47.333 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:52:47.333 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:52:55.335 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8000 2020-05-24T13:52:55.388 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:52:55.388 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:52:55.599 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 1014 0 --:--:-- --:--:-- --:--:-- 1014 2020-05-24T13:52:55.600 INFO:teuthology.orchestra.run.smithi050.stdout:anonymous 2020-05-24T13:52:55.600 INFO:tasks.rgw_multisite:creating admin user.. 2020-05-24T13:52:55.601 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:55.601 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:55.602 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid realm-admin --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Realm Admin' --system --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:52:59.013 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"realm-admin","display_name":"Realm Admin","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"realm-admin","access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","system":"true","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2020-05-24T13:52:59.589 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2020-05-24T13:52:59.590 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi050.front.sepia.ceph.com:8000', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:59.590 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi050.front.sepia.ceph.com:8000', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:52:59.591 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi050.front.sepia.ceph.com:8000 --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:01.181 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","name":"test-realm","current_period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","epoch":2} 2020-05-24T13:53:01.182 INFO:tasks.rgw_multisite:creating zone.. 2020-05-24T13:53:01.182 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--endpoints', 'http://smithi160.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:01.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--endpoints', 'http://smithi160.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:01.183 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --endpoints http://smithi160.front.sepia.ceph.com:8000 --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:02.386 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:53:02.387 7fea6cf7d840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T13:53:02.387 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:53:02.388 7fea6cf7d840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T13:53:02.534 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T13:53:02.535 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool create test-zone2.rgw.buckets.data 64 64 --cluster c2 2020-05-24T13:53:04.269 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'test-zone2.rgw.buckets.data' created 2020-05-24T13:53:04.283 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool application enable test-zone2.rgw.buckets.data rgw --cluster c2 2020-05-24T13:53:05.213 INFO:teuthology.orchestra.run.smithi160.stderr:enabled application 'rgw' on pool 'test-zone2.rgw.buckets.data' 2020-05-24T13:53:05.225 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:05.225 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:05.226 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:09.502 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:53:09.503 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:09.504 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:09.504 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:29.066 INFO:teuthology.orchestra.run.smithi160.stdout:Sending period to new master zone d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T13:53:33.660 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","epoch":2,"predecessor_uuid":"07b6e218-7bbb-4b4a-8656-8ecffcf68225","sync_status":[],"period_map":{"id":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":2} 2020-05-24T13:53:33.661 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2020-05-24T13:53:33.662 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T13:53:33.677 INFO:tasks.rgw.c2.client.0.smithi160.stdout:2020-05-24 13:53:33.667 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends civetweb port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground (PID: 12934) UID: 0 2020-05-24T13:53:33.678 INFO:tasks.rgw.c2.client.0.smithi160.stdout:2020-05-24 13:53:33.670 4049040 -1 shutting down 2020-05-24T13:53:39.763 INFO:tasks.rgw.c2.client.0:Stopped 2020-05-24T13:53:39.764 INFO:tasks.rgw.c2.client.0:Restarting daemon 2020-05-24T13:53:39.765 INFO:teuthology.orchestra.run.smithi160:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2020-05-24T13:53:39.769 INFO:tasks.rgw.c2.client.0:Started 2020-05-24T13:53:39.770 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8000 2020-05-24T13:53:39.847 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:39.848 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:39.851 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi160.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:53:39.852 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:53:40.854 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8000 2020-05-24T13:53:40.939 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:40.940 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:40.943 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi160.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:53:40.944 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:53:42.946 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8000 2020-05-24T13:53:43.018 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:43.020 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:43.023 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi160.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:53:43.023 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:53:47.025 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8000 2020-05-24T13:53:47.100 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:47.100 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:47.105 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi160.front.sepia.ceph.com:8000; Connection refused 2020-05-24T13:53:47.105 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:53:55.106 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8000 2020-05-24T13:53:55.152 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:53:55.153 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:53:55.407 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 841 0 --:--:-- --:--:-- --:--:-- 842 2020-05-24T13:53:55.408 INFO:teuthology.orchestra.run.smithi160.stdout:anonymous 2020-05-24T13:53:55.408 INFO:tasks.rgw_multisite:creating zone.. 2020-05-24T13:53:55.422 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--endpoints', 'http://smithi050.front.sepia.ceph.com:8001', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:55.423 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--endpoints', 'http://smithi050.front.sepia.ceph.com:8001', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:55.431 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --endpoints http://smithi050.front.sepia.ceph.com:8001 --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:53:56.916 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:53:56.922 7fe58be35840 0 failed reading obj info from .rgw.root:zone_info.8de99cfe-4678-427d-bca8-2f77415e7812: (2) No such file or directory 2020-05-24T13:53:56.917 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 13:53:56.922 7fe58be35840 0 WARNING: could not read zone params for zone id=8de99cfe-4678-427d-bca8-2f77415e7812 name=test-zone2 2020-05-24T13:53:56.991 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","domain_root":"test-zone3.rgw.meta:root","control_pool":"test-zone3.rgw.control","gc_pool":"test-zone3.rgw.log:gc","lc_pool":"test-zone3.rgw.log:lc","log_pool":"test-zone3.rgw.log","intent_log_pool":"test-zone3.rgw.log:intent","usage_log_pool":"test-zone3.rgw.log:usage","reshard_pool":"test-zone3.rgw.log:reshard","user_keys_pool":"test-zone3.rgw.meta:users.keys","user_email_pool":"test-zone3.rgw.meta:users.email","user_swift_pool":"test-zone3.rgw.meta:users.swift","user_uid_pool":"test-zone3.rgw.meta:users.uid","otp_pool":"test-zone3.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone3.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone3.rgw.buckets.data"}},"data_extra_pool":"test-zone3.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T13:53:56.992 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool create test-zone3.rgw.buckets.data 64 64 --cluster c1 2020-05-24T13:53:58.629 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'test-zone3.rgw.buckets.data' created 2020-05-24T13:53:58.645 INFO:teuthology.orchestra.run.smithi050:> sudo ceph osd pool application enable test-zone3.rgw.buckets.data rgw --cluster c1 2020-05-24T13:53:59.617 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rgw' on pool 'test-zone3.rgw.buckets.data' 2020-05-24T13:53:59.631 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:59.631 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:53:59.632 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:54:04.704 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","domain_root":"test-zone3.rgw.meta:root","control_pool":"test-zone3.rgw.control","gc_pool":"test-zone3.rgw.log:gc","lc_pool":"test-zone3.rgw.log:lc","log_pool":"test-zone3.rgw.log","intent_log_pool":"test-zone3.rgw.log:intent","usage_log_pool":"test-zone3.rgw.log:usage","reshard_pool":"test-zone3.rgw.log:reshard","user_keys_pool":"test-zone3.rgw.meta:users.keys","user_email_pool":"test-zone3.rgw.meta:users.email","user_swift_pool":"test-zone3.rgw.meta:users.swift","user_uid_pool":"test-zone3.rgw.meta:users.uid","otp_pool":"test-zone3.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone3.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone3.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone3.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":""} 2020-05-24T13:54:04.705 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:54:04.705 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:54:04.706 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:54:35.708 INFO:teuthology.orchestra.run.smithi050.stdout:Sending period to new master zone d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T13:54:38.732 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","epoch":3,"predecessor_uuid":"07b6e218-7bbb-4b4a-8656-8ecffcf68225","sync_status":[],"period_map":{"id":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":2} 2020-05-24T13:54:38.733 DEBUG:tasks.rgw.c1.client.1:waiting for process to exit 2020-05-24T13:54:38.734 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T13:54:38.751 INFO:tasks.rgw.c1.client.1.smithi050.stdout:2020-05-24 13:54:38.746 1b9a3700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends civetweb port=8001 -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground (PID: 13117) UID: 0 2020-05-24T13:54:38.752 INFO:tasks.rgw.c1.client.1.smithi050.stdout:2020-05-24 13:54:38.753 4049040 -1 shutting down 2020-05-24T13:54:44.837 INFO:tasks.rgw.c1.client.1:Stopped 2020-05-24T13:54:44.838 INFO:tasks.rgw.c1.client.1:Restarting daemon 2020-05-24T13:54:44.838 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2020-05-24T13:54:44.842 INFO:tasks.rgw.c1.client.1:Started 2020-05-24T13:54:44.843 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T13:54:44.891 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:44.891 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:44.894 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:54:44.895 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:54:45.896 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T13:54:45.949 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:45.950 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:45.953 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:54:45.953 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:54:47.955 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T13:54:48.011 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:48.011 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:48.011 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:54:48.012 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:54:52.013 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T13:54:52.061 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:54:52.063 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:54:52.066 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:54:52.066 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:55:00.068 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T13:55:00.118 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:00.119 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:00.428 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 690 0 --:--:-- --:--:-- --:--:-- 692 2020-05-24T13:55:00.428 INFO:teuthology.orchestra.run.smithi050.stdout:anonymous 2020-05-24T13:55:00.429 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2020-05-24T13:55:00.429 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi050.front.sepia.ceph.com:8000', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--url', 'http://smithi050.front.sepia.ceph.com:8000', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:00.430 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi050.front.sepia.ceph.com:8000', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--url', 'http://smithi050.front.sepia.ceph.com:8000', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:00.430 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi050.front.sepia.ceph.com:8000 --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --url http://smithi050.front.sepia.ceph.com:8000 --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:55:01.311 INFO:tasks.rgw.c2.client.0.smithi160.stdout:2020-05-24 13:55:01.305 149889700 -1 rgw realm watcher: RGWRealmWatcher::handle_error oid=realms.96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d.control err=-107 2020-05-24T13:55:01.828 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","name":"test-realm","current_period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","epoch":2} 2020-05-24T13:55:01.829 INFO:tasks.rgw_multisite:creating zone.. 2020-05-24T13:55:01.830 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--endpoints', 'http://smithi160.front.sepia.ceph.com:8001', '--tier-type', 'pubsub', '--sync-from-all=0', '--sync-from', 'test-zone1', '--tier-config', 'start_with_full_sync=false,event_retention_days=7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:01.830 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--endpoints', 'http://smithi160.front.sepia.ceph.com:8001', '--tier-type', 'pubsub', '--sync-from-all=0', '--sync-from', 'test-zone1', '--tier-config', 'start_with_full_sync=false,event_retention_days=7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:01.831 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --endpoints http://smithi160.front.sepia.ceph.com:8001 --tier-type pubsub --sync-from-all=0 --sync-from test-zone1 --tier-config start_with_full_sync=false,event_retention_days=7 --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:55:03.172 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:55:03.175 7fb125890840 0 failed reading obj info from .rgw.root:zone_info.88888d62-7b54-4b85-bd95-327afc14fb31: (2) No such file or directory 2020-05-24T13:55:03.173 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:55:03.175 7fb125890840 0 WARNING: could not read zone params for zone id=88888d62-7b54-4b85-bd95-327afc14fb31 name=test-zone3 2020-05-24T13:55:03.196 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:55:03.199 7fb125890840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T13:55:03.197 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 13:55:03.199 7fb125890840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T13:55:03.265 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","domain_root":"test-zone4.rgw.meta:root","control_pool":"test-zone4.rgw.control","gc_pool":"test-zone4.rgw.log:gc","lc_pool":"test-zone4.rgw.log:lc","log_pool":"test-zone4.rgw.log","intent_log_pool":"test-zone4.rgw.log:intent","usage_log_pool":"test-zone4.rgw.log:usage","reshard_pool":"test-zone4.rgw.log:reshard","user_keys_pool":"test-zone4.rgw.meta:users.keys","user_email_pool":"test-zone4.rgw.meta:users.email","user_swift_pool":"test-zone4.rgw.meta:users.swift","user_uid_pool":"test-zone4.rgw.meta:users.uid","otp_pool":"test-zone4.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone4.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone4.rgw.buckets.data"}},"data_extra_pool":"test-zone4.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","tier_config":{"event_retention_days":7,"start_with_full_sync":false},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T13:55:03.266 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool create test-zone4.rgw.buckets.data 64 64 --cluster c2 2020-05-24T13:55:04.188 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'test-zone4.rgw.buckets.data' created 2020-05-24T13:55:04.201 INFO:teuthology.orchestra.run.smithi160:> sudo ceph osd pool application enable test-zone4.rgw.buckets.data rgw --cluster c2 2020-05-24T13:55:05.186 INFO:teuthology.orchestra.run.smithi160.stderr:enabled application 'rgw' on pool 'test-zone4.rgw.buckets.data' 2020-05-24T13:55:05.205 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:05.206 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:05.206 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:55:08.896 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","domain_root":"test-zone4.rgw.meta:root","control_pool":"test-zone4.rgw.control","gc_pool":"test-zone4.rgw.log:gc","lc_pool":"test-zone4.rgw.log:lc","log_pool":"test-zone4.rgw.log","intent_log_pool":"test-zone4.rgw.log:intent","usage_log_pool":"test-zone4.rgw.log:usage","reshard_pool":"test-zone4.rgw.log:reshard","user_keys_pool":"test-zone4.rgw.meta:users.keys","user_email_pool":"test-zone4.rgw.meta:users.email","user_swift_pool":"test-zone4.rgw.meta:users.swift","user_uid_pool":"test-zone4.rgw.meta:users.uid","otp_pool":"test-zone4.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone4.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone4.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone4.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","tier_config":{"event_retention_days":7,"start_with_full_sync":false},"realm_id":""} 2020-05-24T13:55:08.897 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:08.897 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:55:08.902 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T13:55:40.251 INFO:teuthology.orchestra.run.smithi160.stdout:Sending period to new master zone d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T13:55:43.721 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","epoch":4,"predecessor_uuid":"07b6e218-7bbb-4b4a-8656-8ecffcf68225","sync_status":[],"period_map":{"id":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":2} 2020-05-24T13:55:43.722 DEBUG:tasks.rgw.c2.client.1:waiting for process to exit 2020-05-24T13:55:43.723 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T13:55:43.741 INFO:tasks.rgw.c2.client.1.smithi160.stdout:2020-05-24 13:55:43.731 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends civetweb port=8001 -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground (PID: 12936) UID: 0 2020-05-24T13:55:43.742 INFO:tasks.rgw.c2.client.1.smithi160.stdout:2020-05-24 13:55:43.742 4049040 -1 shutting down 2020-05-24T13:55:49.826 INFO:tasks.rgw.c2.client.1:Stopped 2020-05-24T13:55:49.826 INFO:tasks.rgw.c2.client.1:Restarting daemon 2020-05-24T13:55:49.827 INFO:teuthology.orchestra.run.smithi160:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c2.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2020-05-24T13:55:49.830 INFO:tasks.rgw.c2.client.1:Started 2020-05-24T13:55:49.831 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8001 2020-05-24T13:55:49.895 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:49.895 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:49.901 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi160.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:55:49.914 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:55:50.915 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8001 2020-05-24T13:55:50.965 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:50.966 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:50.970 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi160.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:55:50.971 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:55:52.972 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8001 2020-05-24T13:55:53.023 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:53.023 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:53.027 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi160.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:55:53.028 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:55:57.029 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8001 2020-05-24T13:55:57.076 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:55:57.077 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:55:57.081 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi160.front.sepia.ceph.com:8001; Connection refused 2020-05-24T13:55:57.082 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T13:56:05.086 INFO:teuthology.orchestra.run.smithi160:> curl http://smithi160.front.sepia.ceph.com:8001 2020-05-24T13:56:05.132 INFO:teuthology.orchestra.run.smithi160.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T13:56:05.133 INFO:teuthology.orchestra.run.smithi160.stderr: Dload Upload Total Spent Left Speed 2020-05-24T13:56:05.217 INFO:teuthology.orchestra.run.smithi160.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 134 100 134 0 0 1580 0 --:--:-- --:--:-- --:--:-- 1595 2020-05-24T13:56:05.217 INFO:teuthology.orchestra.run.smithi160.stdout:{"Code":"MethodNotAllowed","RequestId":"tx000000000000000000001-005eca7cf5-110c-test-zone4","HostId":"110c-test-zone4-test-zonegroup"} 2020-05-24T13:56:05.218 INFO:tasks.rgw_multisite:rgw multisite configuration completed 2020-05-24T13:56:05.218 INFO:teuthology.run_tasks:Running task rgw-multisite-tests... 2020-05-24T13:56:05.299 INFO:tasks.rgw_multisite_tests:creating test user.. 2020-05-24T13:56:05.299 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'rgw-multisite-test-user', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Multisite Test User', '--gen-access-key', '--gen-secret', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:56:05.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'rgw-multisite-test-user', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Multisite Test User', '--gen-access-key', '--gen-secret', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T13:56:05.300 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid rgw-multisite-test-user --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Multisite Test User' --gen-access-key --gen-secret --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T13:56:08.538 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"rgw-multisite-test-user","display_name":"Multisite Test User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"rgw-multisite-test-user","access_key":"F5LSZ1G3C982N0OF577K","secret_key":"TbdNJP1aBpdoDayuDevuUdRsOPs9BxOdmCNv6KgU"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2020-05-24T13:56:09.294 INFO:rgw_multi.tests:meta checkpoint 2020-05-24T13:56:09.295 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:09.296 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:09.296 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:11.977 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T13:56:12.857 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T13:56:12.858 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:56:12.858 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:12.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:12.859 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:15.698 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:15.698 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1590328568.276896_39.1 target= 2020-05-24T13:56:20.699 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:20.700 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:20.700 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:23.666 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:23.666 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:56:23.667 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:56:23.667 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:23.668 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:23.668 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:26.391 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:26.392 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328568.276896_39.1 target= 2020-05-24T13:56:31.393 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:31.394 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:31.394 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:34.470 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:34.471 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328568.276896_39.1 target= 2020-05-24T13:56:39.472 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:39.472 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:39.473 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:42.180 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:42.181 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328568.276896_39.1 target= 2020-05-24T13:56:47.182 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:47.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:47.184 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:50.595 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:50.595 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328568.276896_39.1 target= 2020-05-24T13:56:55.597 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:55.598 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:56:55.598 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:56:58.575 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:56:58.575 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328568.276896_39.1 target= 2020-05-24T13:57:03.576 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:03.577 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:03.577 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:06.584 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:06.589 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328568.276896_39.1 target= 2020-05-24T13:57:11.590 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:11.591 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:11.591 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:14.611 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:14.612 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=0 master=1_1590328568.276896_39.1 target= 2020-05-24T13:57:19.613 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:19.614 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:19.615 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:22.193 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:22.193 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:57:22.194 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:57:22.194 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:22.194 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:22.195 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:23.613 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:08:57.669 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T13:57:23.694 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:08:57.749 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T13:57:24.255 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:08:58.310 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T13:57:25.005 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:25.006 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:57:25.007 INFO:rgw_multi.tests:meta checkpoint 2020-05-24T13:57:25.007 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:25.008 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:25.008 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:27.038 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T13:57:27.565 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T13:57:27.565 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:57:27.566 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:27.566 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:27.567 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:30.127 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:30.127 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:57:30.128 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:57:30.128 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:30.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:30.129 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:32.920 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:32.921 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:57:32.922 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:57:32.922 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:32.922 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:57:32.923 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:57:35.634 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:57:35.634 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:57:35.635 INFO:tasks.rgw_multisite_tests:running rgw multisite tests on 'tasks.rgw_multi.tests' with args=[] 2020-05-24T13:57:35.651 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-1 2020-05-24T13:57:46.755 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-2 2020-05-24T13:57:59.762 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-3 2020-05-24T13:58:01.964 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:01.965 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:01.966 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T13:58:05.025 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590328681.510269_55.1","last_update":"2020-05-24 13:58:01.510269Z"}] 2020-05-24T13:58:05.026 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:58:05.026 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:05.027 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:05.027 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:07.468 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328681.510269_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:01.510269Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:07.468 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:58:07.469 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:58:07.470 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:07.471 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:07.471 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:10.401 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328681.510269_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:01.510269Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:10.402 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:58:10.403 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:58:10.403 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:10.404 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:10.404 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:13.202 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328681.510269_55.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:01.510269Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:13.203 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:58:15.400 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_create ... ok 2020-05-24T13:58:15.401 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-4 2020-05-24T13:58:16.508 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-5 2020-05-24T13:58:18.536 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-6 2020-05-24T13:58:20.418 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:20.418 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:20.419 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:22.630 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T13:58:23.307 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590328700.029897_67.1","last_update":"2020-05-24 13:58:20.029897Z"}] 2020-05-24T13:58:23.308 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:58:23.308 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:23.309 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:23.310 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:25.839 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328700.029897_67.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:20.029897Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:25.840 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:58:25.841 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:58:25.841 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:25.842 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:25.842 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:28.700 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328700.029897_67.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:20.029897Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:28.703 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:58:28.703 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:58:28.704 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:28.704 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:28.705 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:30.065 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:08:57.312 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T13:58:30.189 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:08:57.436 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T13:58:30.634 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:08:57.881 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T13:58:31.514 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328700.029897_67.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:20.029897Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:31.514 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:58:50.954 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:50.955 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:50.956 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:53.073 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T13:58:53.665 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590328728.582516_125.1","last_update":"2020-05-24 13:58:48.582516Z"}] 2020-05-24T13:58:53.665 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:58:53.666 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:53.666 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:53.667 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:56.081 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328728.582516_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:48.582516Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:56.082 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:58:56.082 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:58:56.083 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:56.084 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:56.084 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:58:58.810 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328728.582516_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:48.582516Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:58:58.811 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:58:58.812 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:58:58.813 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:58.814 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:58:58.814 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:01.266 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328728.582516_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:58:48.582516Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:01.267 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:59:04.510 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_recreate ... ok 2020-05-24T13:59:04.511 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-7 2020-05-24T13:59:05.441 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-8 2020-05-24T13:59:07.981 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-9 2020-05-24T13:59:10.815 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:10.816 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:10.817 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:12.972 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T13:59:13.655 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590328750.444965_140.1","last_update":"2020-05-24 13:59:10.444965Z"}] 2020-05-24T13:59:13.655 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:59:13.656 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:13.656 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:13.657 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:16.091 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328750.444965_140.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:10.444965Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:16.092 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:59:16.092 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:59:16.093 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:16.093 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:16.094 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:18.919 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328750.444965_140.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:10.444965Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:18.919 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:59:18.921 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:59:18.923 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:18.925 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:18.934 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:21.531 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328750.444965_140.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:10.444965Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:21.532 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:59:27.845 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:27.847 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:27.848 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:29.934 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T13:59:30.572 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T13:59:30.573 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T13:59:30.574 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:30.574 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:30.575 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:33.140 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:33.140 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T13:59:33.141 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T13:59:33.142 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:33.142 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:33.143 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:36.032 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:36.032 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T13:59:36.033 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T13:59:36.033 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:36.034 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:36.035 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T13:59:38.726 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T13:59:38.727 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T13:59:40.447 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_remove ... ok 2020-05-24T13:59:40.448 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-10 2020-05-24T13:59:41.363 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-11 2020-05-24T13:59:42.982 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-12 2020-05-24T13:59:56.452 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:56.453 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T13:59:56.453 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:01.558 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T14:00:02.225 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590328783.837755_47.1","last_update":"2020-05-24 13:59:43.837755Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T14:00:02.225 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:00:02.226 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:02.227 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:02.227 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:04.943 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590328783.837755_47.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:43.837755Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:04.944 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:00:04.944 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:00:04.945 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:04.945 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:04.946 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:07.585 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590328783.837755_47.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:43.837755Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:07.586 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:00:07.587 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:00:07.587 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:07.588 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:07.588 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:10.557 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590328783.837755_47.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:43.837755Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:00:10.558 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:00:10.558 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:10.559 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:10.560 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:13.470 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.8.5"}]} 2020-05-24T14:00:13.470 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-10 2020-05-24T14:00:13.471 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:13.471 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:13.472 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-10 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:16.180 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.8.5"}}}] 2020-05-24T14:00:16.181 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-10 2020-05-24T14:00:16.181 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-10 zones={test-zone2, test-zone1} 2020-05-24T14:00:16.582 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:00:16.637 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:16.638 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:16.638 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:16.638 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:16.639 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:00:16.744 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:16.745 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:16.746 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:16.746 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:16.852 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:17.023 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:17.302 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:17.518 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:17.674 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:17.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:18.218 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:18.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:18.785 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:19.074 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:19.272 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:19.531 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:19.798 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-10 zones={test-zone2, test-zone1} 2020-05-24T14:00:19.799 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:19.799 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:19.800 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:22.628 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.8.5"}]} 2020-05-24T14:00:22.629 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-10 2020-05-24T14:00:22.630 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:22.631 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:22.632 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:25.360 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.8.5"}}}] 2020-05-24T14:00:25.361 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-10 2020-05-24T14:00:25.362 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-10 zones={test-zone3, test-zone1} 2020-05-24T14:00:25.992 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:00:26.094 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:26.095 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:26.095 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:26.096 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:26.096 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:00:26.170 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:26.171 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:26.171 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:26.172 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:26.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:26.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:26.801 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:27.053 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:27.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:27.549 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:27.728 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:27.893 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:28.248 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:28.535 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:28.686 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:29.024 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:29.209 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-10 zones={test-zone3, test-zone1} 2020-05-24T14:00:29.210 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:29.211 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-10', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:29.211 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-10 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:31.552 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.8.5"}]} 2020-05-24T14:00:31.552 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-10 2020-05-24T14:00:31.553 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:31.553 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-10', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:31.554 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-10 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:34.076 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.8.5"}}}] 2020-05-24T14:00:34.077 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-10 2020-05-24T14:00:34.078 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:34.078 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:34.079 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:36.435 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000009.10.3"}]} 2020-05-24T14:00:36.436 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-11 2020-05-24T14:00:36.436 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:36.437 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:36.437 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:39.244 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000009.10.3"}}}] 2020-05-24T14:00:39.245 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-11 2020-05-24T14:00:39.246 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-11 zones={test-zone1, test-zone2} 2020-05-24T14:00:40.376 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:00:41.202 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:41.203 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:41.203 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:41.204 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:41.204 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:00:41.284 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:41.285 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:41.285 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:41.286 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:41.425 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:41.597 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:41.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:42.206 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:42.488 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:42.907 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:43.122 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:43.394 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:43.670 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:43.887 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:44.061 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:44.307 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:44.633 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-11 zones={test-zone1, test-zone2} 2020-05-24T14:00:44.634 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:44.635 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:44.635 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:46.963 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000009.10.3"}]} 2020-05-24T14:00:46.964 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-11 2020-05-24T14:00:46.964 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:46.965 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:46.965 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:49.831 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000009.10.3"}}}] 2020-05-24T14:00:49.832 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-11 2020-05-24T14:00:49.833 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-11 zones={test-zone3, test-zone2} 2020-05-24T14:00:50.834 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:00:50.919 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:50.920 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:50.920 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:50.920 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:50.921 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:00:50.977 DEBUG:tasks.rgw_multi.zone_rados:o=& 2020-05-24T14:00:50.977 DEBUG:tasks.rgw_multi.zone_rados:o=: 2020-05-24T14:00:50.978 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2020-05-24T14:00:50.978 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:00:51.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:51.256 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:51.550 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2020-05-24T14:00:51.758 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:51.970 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:52.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2020-05-24T14:00:52.487 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:52.691 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:52.994 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2020-05-24T14:00:53.194 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:53.366 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:53.666 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:00:53.897 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-11 zones={test-zone3, test-zone2} 2020-05-24T14:00:53.898 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:53.898 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:53.899 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:56.341 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000009.10.3"}]} 2020-05-24T14:00:56.521 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-11 2020-05-24T14:00:56.522 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:56.522 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:00:56.523 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:00:59.169 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:00:59.170 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:01:04.171 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:04.172 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:04.173 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:06.713 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:06.714 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:01:11.716 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:11.716 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:11.717 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:14.357 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:14.358 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:01:19.359 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:19.360 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:19.361 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:22.072 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:22.072 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:01:27.074 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:27.075 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:27.076 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:29.404 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:29.404 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:01:34.405 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:34.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:34.413 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:37.455 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:37.456 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:01:42.457 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:42.458 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:42.459 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:45.049 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:45.050 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:01:50.051 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:50.052 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:50.053 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:01:52.618 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:01:52.618 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:01:57.620 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:57.621 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:01:57.622 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:00.147 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:00.147 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:02:05.148 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:05.149 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:05.150 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:07.758 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:07.758 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:02:12.759 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:12.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:12.762 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:15.444 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:15.445 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:02:20.446 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:20.447 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:20.447 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:23.134 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:23.134 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:02:28.136 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:28.137 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:28.138 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:30.939 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:30.939 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:02:35.941 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:35.942 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:35.943 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:39.070 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:39.070 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:02:44.072 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:44.073 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:44.073 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:46.584 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:46.584 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:02:51.586 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:51.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:51.587 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:02:54.116 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:02:54.117 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:02:59.118 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:59.119 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:02:59.120 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:01.739 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:01.740 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:03:06.741 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:06.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:06.743 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:09.052 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:09.052 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:03:14.054 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:14.055 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:14.056 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:17.065 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:17.066 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:03:22.067 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:22.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:22.069 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:24.842 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:24.842 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:03:29.843 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:29.844 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:29.845 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:32.411 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:32.412 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:03:37.413 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:37.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:37.414 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:40.103 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:40.104 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:03:45.105 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:45.105 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:45.106 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:47.598 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:47.599 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:03:52.601 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:52.602 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:03:52.602 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:03:55.280 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:03:55.281 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:04:00.282 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:00.283 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:00.284 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:03.071 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:03.071 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:04:08.073 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:08.074 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:08.075 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:10.831 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:10.832 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:04:15.833 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:15.834 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:15.835 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:18.452 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:18.453 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:04:23.458 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:23.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:23.459 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:26.086 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:26.087 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:04:31.089 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:31.090 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:31.091 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:33.729 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:33.729 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:04:38.731 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:38.732 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:38.733 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:41.277 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:41.278 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:04:46.279 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:46.280 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:46.281 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:49.277 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:49.278 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:04:54.284 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:54.284 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:04:54.285 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:04:57.215 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:04:57.216 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:05:02.217 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:02.217 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:02.218 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:04.874 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:04.875 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:05:09.878 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:09.879 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:09.880 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:12.539 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:12.540 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:05:15.566 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:16:49.630 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:15.567 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:16:49.632 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:15.957 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:16:50.023 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:17.542 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:17.543 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:17.544 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:20.110 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:20.111 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:05:25.113 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:25.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:25.116 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:27.815 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:27.816 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:05:32.818 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:32.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:32.820 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:35.625 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:35.626 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:05:40.635 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:40.636 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:40.637 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:43.458 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:43.458 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:05:44.382 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:17:18.449 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:44.390 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:17:18.457 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:44.427 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:17:18.494 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:05:48.459 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:48.460 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:48.461 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:51.207 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:51.208 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:05:56.209 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:56.210 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:05:56.211 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:05:59.309 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:05:59.310 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:06:04.312 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:04.313 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:04.314 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:08.347 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:08.348 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:06:13.349 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:13.351 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:13.352 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:16.281 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:16.282 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:06:21.283 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:21.284 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:21.285 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:23.473 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:23.473 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:06:26.595 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:16:53.848 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:26.598 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:16:53.851 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:26.931 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:16:54.184 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:28.475 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:28.476 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:28.476 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:30.690 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:30.691 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:06:35.693 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:35.694 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:35.695 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:38.120 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:38.120 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:06:43.122 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:43.123 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:43.123 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:45.740 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:45.740 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:06:50.741 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:50.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:50.743 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:06:50.925 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:17:18.178 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:50.943 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:17:18.196 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:50.984 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:17:18.237 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:06:53.368 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:06:53.369 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:06:58.370 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:58.372 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:06:58.373 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:01.190 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:01.191 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:07:06.192 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:06.193 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:06.194 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:09.375 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:09.376 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:07:14.377 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:14.378 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:14.379 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:16.910 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:16.910 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:07:21.912 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:21.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:21.914 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:24.758 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:24.759 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:07:29.760 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:29.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:29.762 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:32.338 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:32.339 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:07:37.340 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:37.341 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:37.342 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:40.361 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:40.362 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:07:45.363 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:45.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:45.365 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:47.895 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:47.895 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:07:52.897 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:52.898 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:07:52.898 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:07:55.623 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:07:55.624 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:08:00.626 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:00.627 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:00.627 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:03.346 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:03.347 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:08:08.348 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:08.349 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:08.350 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:11.319 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:11.320 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:08:16.322 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:16.323 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:16.324 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:19.360 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:19.360 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:08:24.362 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:24.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:24.364 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:27.043 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:27.044 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:08:32.045 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:32.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:32.054 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-11 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:34.958 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:08:34.958 WARNING:rgw_multi.tests:bucket ajxnhd-11 zone test-zone4 behind zone test-zone2: shard=0 master=00000000009.10.3 target= 2020-05-24T14:08:39.960 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_object_sync ... FAIL 2020-05-24T14:08:39.962 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-13 2020-05-24T14:08:40.763 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-14 2020-05-24T14:08:43.572 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-15 2020-05-24T14:08:48.220 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:48.220 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:48.221 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:53.706 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T14:08:54.602 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590329324.279876_65.1","last_update":"2020-05-24 14:08:44.279876Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T14:08:54.603 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:08:54.604 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:54.605 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:54.605 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:08:57.313 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590329324.279876_65.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:08:44.279876Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:08:57.314 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:08:57.314 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:08:57.315 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:57.315 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:08:57.316 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:00.428 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590329324.279876_65.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:08:44.279876Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:09:00.429 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:09:00.430 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:09:00.431 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:00.431 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:00.432 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:02.965 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590329324.279876_65.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:08:44.279876Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:09:02.966 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:09:02.967 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:02.968 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:02.969 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:05.676 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:09:05.677 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-13 2020-05-24T14:09:05.678 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:05.678 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:05.679 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:08.335 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:09:08.336 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-13 2020-05-24T14:09:08.336 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-13 zones={test-zone2, test-zone1} 2020-05-24T14:09:09.558 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:09:09.655 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:09.656 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:09:09.731 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:09.821 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:10.037 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:10.373 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:10.607 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-13 zones={test-zone2, test-zone1} 2020-05-24T14:09:10.608 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:10.609 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:10.609 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:13.198 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:09:13.198 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-13 2020-05-24T14:09:13.199 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:13.199 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:13.200 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:16.059 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:09:16.059 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-13 2020-05-24T14:09:16.060 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-13 zones={test-zone3, test-zone1} 2020-05-24T14:09:17.034 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:09:17.086 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:17.086 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:09:17.160 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:17.294 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:17.451 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:17.747 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:17.983 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-13 zones={test-zone3, test-zone1} 2020-05-24T14:09:17.984 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:17.984 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-13', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:17.985 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-13 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:20.489 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:09:20.489 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-13 2020-05-24T14:09:20.490 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:20.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-13', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:20.491 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-13 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:23.314 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:09:23.315 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-13 2020-05-24T14:09:23.315 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:23.322 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:23.323 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:25.752 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T14:09:25.753 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-14 2020-05-24T14:09:25.753 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:25.754 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:25.754 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:28.344 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.3.5"}}}] 2020-05-24T14:09:28.345 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-14 2020-05-24T14:09:28.345 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-14 zones={test-zone1, test-zone2} 2020-05-24T14:09:30.261 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:09:30.340 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:30.341 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:09:30.411 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:30.501 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:30.638 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:30.899 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:31.159 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-14 zones={test-zone1, test-zone2} 2020-05-24T14:09:31.160 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:31.160 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:31.161 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:33.688 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T14:09:33.688 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-14 2020-05-24T14:09:33.689 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:33.690 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:33.690 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:36.586 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"myobj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.3.5"}}}] 2020-05-24T14:09:36.587 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-14 2020-05-24T14:09:36.587 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-14 zones={test-zone3, test-zone2} 2020-05-24T14:09:37.326 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T14:09:37.394 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:37.395 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T14:09:37.524 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2020-05-24T14:09:37.615 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:37.789 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:38.100 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2020-05-24T14:09:38.276 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-14 zones={test-zone3, test-zone2} 2020-05-24T14:09:38.276 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:38.277 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:38.278 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:40.761 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.3.5"}]} 2020-05-24T14:09:40.762 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-14 2020-05-24T14:09:40.762 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:40.763 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:40.764 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:43.227 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:09:43.227 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:09:48.228 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:48.229 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:48.230 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:50.813 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:09:50.814 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:09:55.816 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:55.818 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:09:55.819 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:09:58.581 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:09:58.581 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:03.583 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:03.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:03.585 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:06.247 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:06.248 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:11.250 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:11.250 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:11.251 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:14.250 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:14.251 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:19.252 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:19.253 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:19.254 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:21.806 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:21.807 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:26.808 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:26.809 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:26.810 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:29.378 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:29.379 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:34.380 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:34.381 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:34.382 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:37.676 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:37.676 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:42.677 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:42.678 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:42.679 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:45.371 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:45.372 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:50.373 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:50.373 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:50.374 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:10:53.569 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:10:53.569 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:10:58.571 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:58.572 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:10:58.572 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:01.367 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:01.367 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:06.369 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:06.370 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:06.371 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:08.817 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:08.818 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:13.820 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:13.821 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:13.822 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:16.473 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:16.474 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:21.487 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:21.488 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:21.490 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:24.422 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:24.423 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:29.424 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:29.425 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:29.426 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:32.046 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:32.047 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:37.048 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:37.049 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:37.050 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:39.804 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:39.805 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:44.806 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:44.807 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:44.808 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:47.497 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:47.498 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:11:52.499 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:52.500 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:11:52.501 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:11:55.340 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:11:55.341 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:00.342 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:00.343 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:00.344 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:03.232 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:03.233 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:08.234 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:08.235 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:08.236 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:10.173 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:23:44.243 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:12:10.174 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:23:44.245 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:12:10.442 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:23:44.514 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:12:10.884 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:10.884 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:15.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:15.886 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:15.887 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:18.831 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:18.831 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:23.833 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:23.834 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:23.834 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:26.612 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:26.613 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:31.614 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:31.615 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:31.616 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:34.349 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:34.350 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:39.351 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:39.352 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:39.353 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:42.137 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:42.138 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:47.140 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:47.140 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:47.141 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:49.931 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:49.933 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:12:54.935 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:54.936 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:12:54.937 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:12:58.018 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:12:58.018 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:03.020 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:03.021 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:03.021 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:05.923 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:05.923 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:10.925 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:10.925 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:10.926 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:13.758 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:13.759 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:18.760 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:18.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:18.761 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:21.509 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:21.509 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:23.025 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:23:50.269 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:13:23.027 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:23:50.271 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:13:23.351 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:23:50.595 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:13:26.511 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:26.512 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:26.513 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:29.516 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:29.517 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:34.518 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:34.519 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:34.520 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:37.397 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:37.403 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:42.405 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:42.406 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:42.407 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:45.074 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:45.074 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:50.075 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:50.076 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:50.077 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:13:52.930 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:13:52.930 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:13:57.932 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:57.933 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:13:57.934 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:00.582 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:00.583 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:05.486 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:25:39.557 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:14:05.487 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:25:39.559 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:14:05.565 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:25:39.636 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:14:05.584 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:05.585 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:05.586 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:08.493 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:08.493 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:13.494 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:13.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:13.496 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:16.142 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:16.142 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:21.144 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:21.145 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:21.146 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:23.813 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:23.814 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:28.816 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:28.817 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:28.818 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:31.668 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:31.669 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:36.670 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:36.671 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:36.672 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:39.460 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:39.461 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:44.463 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:44.464 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:44.464 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:47.125 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:47.126 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:52.127 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:52.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:52.128 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:14:54.942 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:14:54.942 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:14:59.943 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:59.944 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:14:59.945 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:02.626 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:02.627 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:07.628 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:07.629 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:07.629 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:10.351 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:10.352 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:11.706 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:25:38.948 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:15:11.715 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:25:38.957 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:15:11.812 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:25:39.053 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:15:15.353 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:15.354 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:15.354 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:18.634 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:18.635 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:23.636 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:23.637 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:23.638 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:26.206 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:26.206 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:31.208 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:31.209 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:31.209 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:33.846 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:33.847 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:38.849 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:38.849 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:38.850 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:41.318 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:41.318 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:46.320 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:46.321 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:46.321 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:49.529 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:49.530 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:15:54.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:54.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:15:54.533 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:15:57.519 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:15:57.519 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:02.521 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:02.522 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:02.522 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:06.048 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:06.048 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:11.050 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:11.050 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:11.051 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:14.232 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:14.233 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:19.235 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:19.236 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:19.236 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:22.372 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:22.373 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:27.377 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:27.378 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:27.379 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:30.136 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:30.136 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:35.138 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:35.139 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:35.140 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:38.065 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:38.066 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:43.067 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:43.069 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:43.069 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:45.812 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:45.812 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:50.814 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:50.815 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:50.815 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:16:53.812 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:16:53.812 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:16:58.814 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:58.815 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:16:58.815 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:01.836 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:01.836 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:06.837 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:06.838 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:06.839 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:10.069 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:10.069 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:15.071 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:15.072 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:15.072 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:17.905 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:17.906 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:22.907 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:22.908 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-14', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:22.909 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-14 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:25.619 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:17:25.620 WARNING:rgw_multi.tests:bucket ajxnhd-14 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.3.5 target= 2020-05-24T14:17:30.622 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_object_delete ... FAIL 2020-05-24T14:17:30.624 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-16 2020-05-24T14:17:31.702 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-17 2020-05-24T14:17:33.136 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-18 2020-05-24T14:17:38.061 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:38.062 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:38.063 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:40.285 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T14:17:40.887 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590329857.827328_89.1","last_update":"2020-05-24 14:17:37.827328Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T14:17:40.888 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:17:40.889 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:40.889 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:40.890 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:43.550 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590329857.827328_89.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:17:37.827328Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:17:43.551 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:17:43.551 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:17:43.552 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:43.553 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:43.554 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:46.236 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590329857.827328_89.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:17:37.827328Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:17:46.237 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:17:46.237 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:17:46.238 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:46.238 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:46.239 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:48.918 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590329857.827328_89.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:17:37.827328Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:17:48.918 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:17:50.667 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:50.668 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:50.668 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:53.926 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000003.12.10"}]} 2020-05-24T14:17:53.926 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-16 2020-05-24T14:17:53.927 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:53.927 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:53.928 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-16 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:17:57.409 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"mf15rteK5-fWk9BKnexA3ncW81vMa-t","ns":""},"count":1},"inc_marker":{"position":"00000000004.14.9"}}}] 2020-05-24T14:17:57.410 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-16 2020-05-24T14:17:57.410 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:57.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:17:57.411 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:00.587 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000006.18.9"}]} 2020-05-24T14:18:00.587 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-16 2020-05-24T14:18:00.588 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:00.588 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:00.589 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:03.433 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"mf15rteK5-fWk9BKnexA3ncW81vMa-t","ns":""},"count":1},"inc_marker":{"position":"00000000006.18.9"}}}] 2020-05-24T14:18:03.433 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-16 2020-05-24T14:18:03.434 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:03.434 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-16', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:03.435 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-16 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:06.275 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000006.18.9"}]} 2020-05-24T14:18:06.276 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-16 2020-05-24T14:18:06.277 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:06.277 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-16', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:06.279 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-16 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:09.395 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"mf15rteK5-fWk9BKnexA3ncW81vMa-t","ns":""},"count":1},"inc_marker":{"position":"00000000006.18.9"}}}] 2020-05-24T14:18:09.395 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-16 2020-05-24T14:18:10.645 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:10.646 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:10.646 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:13.093 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000003.7.10"}]} 2020-05-24T14:18:13.093 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-17 2020-05-24T14:18:13.094 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:13.094 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:13.095 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:18.109 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"WGETuhNe8GL2VfmTlWy2SWC5kUxQJyR","ns":""},"count":1},"inc_marker":{"position":"00000000007.13.9"}}}] 2020-05-24T14:18:18.110 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-17 2020-05-24T14:18:18.110 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:18.111 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:18.111 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:20.961 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000007.13.9"}]} 2020-05-24T14:18:20.962 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-17 2020-05-24T14:18:20.963 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:20.963 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:20.964 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:24.184 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"WGETuhNe8GL2VfmTlWy2SWC5kUxQJyR","ns":""},"count":1},"inc_marker":{"position":"00000000007.13.9"}}}] 2020-05-24T14:18:24.185 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-17 2020-05-24T14:18:24.186 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:24.187 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:24.187 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:26.649 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000007.13.9"}]} 2020-05-24T14:18:26.649 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-17 2020-05-24T14:18:26.650 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:26.650 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:26.651 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:29.420 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:18:29.420 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:18:34.421 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:34.423 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:34.424 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:37.135 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:18:37.135 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:18:42.137 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:42.138 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:42.138 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:45.014 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:18:45.014 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:18:50.016 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:50.017 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:50.018 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:18:52.784 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:18:52.785 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:18:57.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:57.787 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:18:57.788 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:00.385 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:00.385 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:19:05.387 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:05.388 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:05.389 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:06.590 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:30:40.663 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:19:06.594 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:30:40.666 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:19:06.877 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:30:40.950 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:19:08.219 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:08.220 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:19:13.222 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:13.223 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:13.224 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:15.992 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:15.992 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:19:20.993 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:20.994 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:20.995 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:23.918 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:23.919 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:19:28.920 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:28.921 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:28.922 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:31.902 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:31.902 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:19:36.904 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:36.905 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:36.905 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:39.922 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:39.923 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:19:44.924 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:44.925 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:44.926 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:47.793 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:47.794 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:19:52.795 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:52.796 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:19:52.797 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:19:55.523 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:19:55.524 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:20:00.526 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:00.527 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:00.528 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:03.242 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:03.243 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:20:08.244 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:08.245 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:08.245 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:10.935 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:10.935 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:20:15.938 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:15.939 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:15.940 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:17.270 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:30:44.506 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:20:17.272 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:30:44.508 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:20:17.499 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:30:44.735 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:20:19.054 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:19.054 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:20:24.067 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:24.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:24.068 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:26.945 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:26.946 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:20:31.947 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:31.948 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:31.948 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:34.777 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:34.777 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:20:39.779 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:39.781 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:39.782 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:42.569 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:42.570 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:20:47.571 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:47.572 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:47.573 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:50.603 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:50.604 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:20:55.605 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:55.611 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:20:55.611 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:20:58.168 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:20:58.168 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:21:03.170 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:03.171 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:03.172 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:05.961 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:05.961 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:21:10.965 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:10.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:10.967 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:14.088 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:14.089 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:21:19.090 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:19.091 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:19.092 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:22.173 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:22.174 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:21:27.176 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:27.176 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:27.177 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:30.092 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:30.093 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:21:35.094 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:35.095 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:35.095 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:37.832 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:37.833 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:21:42.834 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:42.835 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:42.836 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:45.643 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:45.644 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:21:50.645 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:50.646 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:50.647 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:21:53.505 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:21:53.506 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:21:58.507 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:58.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:21:58.509 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:01.297 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:01.298 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:22:06.300 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:06.301 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:06.302 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:09.404 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:09.405 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:22:14.406 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:14.407 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:14.407 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:17.232 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:17.233 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:22:22.234 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:22.235 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:22.236 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:24.959 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:24.960 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:22:29.961 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:29.962 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:29.963 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:32.653 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:32.653 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:22:37.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:37.656 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:37.656 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:40.506 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:40.507 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:22:45.507 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:45.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:45.509 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:48.392 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:48.392 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:22:53.394 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:53.395 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:22:53.395 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:22:56.180 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:22:56.181 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:23:01.182 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:01.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:01.184 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:04.336 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:04.337 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:23:09.338 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:09.339 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:09.340 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:11.956 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:11.957 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:23:16.964 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:16.964 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:16.965 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:19.620 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:19.620 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:23:24.621 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:24.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:24.624 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:27.316 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:27.317 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:23:32.318 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:32.319 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:32.320 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:35.188 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:35.189 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:23:40.190 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:40.191 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:40.191 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:43.067 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:43.067 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:23:48.070 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:48.071 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:48.073 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:50.991 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:50.992 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:23:55.993 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:55.994 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:23:55.995 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:23:59.067 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:23:59.067 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:24:04.068 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:04.069 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:04.070 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:07.349 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:07.350 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:24:12.352 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:12.352 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:12.353 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:15.418 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:15.419 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:24:20.420 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:20.421 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:20.422 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:23.217 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:23.218 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:24:28.219 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:28.220 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:28.220 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:31.167 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:31.168 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:24:36.169 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:36.170 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:36.171 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:38.908 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:38.909 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:24:43.909 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:43.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:43.911 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:46.870 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:46.870 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:24:51.872 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:51.873 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:51.874 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:24:54.674 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:24:54.675 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:24:59.680 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:59.681 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:24:59.682 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:02.446 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:02.447 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:25:07.448 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:07.449 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:07.450 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:10.237 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:10.238 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:25:15.239 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:15.240 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:15.242 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:18.074 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:18.076 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:25:23.078 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:23.079 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:23.080 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:25.899 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:25.900 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:25:30.902 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:30.902 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:30.903 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:33.690 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:33.691 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:25:38.692 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:38.693 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:38.694 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:41.595 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:41.596 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:25:46.597 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:46.598 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:46.599 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:49.919 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:49.920 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:25:54.921 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:54.922 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:25:54.923 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:25:58.060 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:25:58.061 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:26:03.062 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:03.063 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:03.063 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:06.632 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:26:06.633 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:26:11.634 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:11.635 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:11.635 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-17 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:14.542 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:26:14.543 WARNING:rgw_multi.tests:bucket ajxnhd-17 zone test-zone4 behind zone test-zone2: shard=0 master=00000000007.13.9 target= 2020-05-24T14:26:19.545 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_versioned_object_incremental_sync ... FAIL 2020-05-24T14:26:20.676 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:20.677 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:20.677 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:22.873 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T14:26:23.613 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590330380.583029_101.1","last_update":"2020-05-24 14:26:20.583029Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T14:26:23.615 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:26:23.615 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:23.616 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:23.616 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:25.895 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590330380.583029_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:26:20.583029Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:26:25.896 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:26:25.896 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:26:25.897 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:25.897 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:25.898 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:28.987 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590330380.583029_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:26:20.583029Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:26:28.988 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:26:28.989 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:26:28.989 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:28.990 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:28.991 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:31.895 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590330380.583029_101.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:26:20.583029Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:26:31.896 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:26:33.226 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:33.227 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:33.228 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:36.772 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000003.6.10"}]} 2020-05-24T14:26:36.773 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-19 2020-05-24T14:26:36.781 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:36.781 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:36.782 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:39.910 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Fbw5IK5hbTUgzS-6WYKKg9zaq6D2Iqr","ns":""},"count":1},"inc_marker":{"position":"00000000005.10.9"}}}] 2020-05-24T14:26:39.911 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-19 2020-05-24T14:26:39.912 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:39.912 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:39.913 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:42.942 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000006.12.9"}]} 2020-05-24T14:26:42.942 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-19 2020-05-24T14:26:42.943 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:42.944 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:42.944 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:45.606 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Fbw5IK5hbTUgzS-6WYKKg9zaq6D2Iqr","ns":""},"count":1},"inc_marker":{"position":"00000000006.12.9"}}}] 2020-05-24T14:26:45.608 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-19 2020-05-24T14:26:45.608 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:45.609 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:45.610 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:48.582 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000006.12.9"}]} 2020-05-24T14:26:48.583 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-19 2020-05-24T14:26:48.583 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:48.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:48.584 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:51.478 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Fbw5IK5hbTUgzS-6WYKKg9zaq6D2Iqr","ns":""},"count":1},"inc_marker":{"position":"00000000006.12.9"}}}] 2020-05-24T14:26:51.479 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-19 2020-05-24T14:26:51.479 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:51.480 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:51.480 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:54.146 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000005.10.9"}]} 2020-05-24T14:26:54.146 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-19 2020-05-24T14:26:54.147 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:54.147 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:54.148 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:56.767 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Fbw5IK5hbTUgzS-6WYKKg9zaq6D2Iqr","ns":""},"count":1},"inc_marker":{"position":"00000000005.10.9"}}}] 2020-05-24T14:26:56.768 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-19 2020-05-24T14:26:56.768 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:56.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:56.770 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:26:59.460 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000005.10.9"}]} 2020-05-24T14:26:59.460 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-19 2020-05-24T14:26:59.461 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:59.462 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:26:59.462 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:02.048 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"dummy","instance":"Fbw5IK5hbTUgzS-6WYKKg9zaq6D2Iqr","ns":""},"count":1},"inc_marker":{"position":"00000000005.10.9"}}}] 2020-05-24T14:27:02.049 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-19 2020-05-24T14:27:02.049 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:02.050 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:02.050 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:04.644 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000005.10.9"}]} 2020-05-24T14:27:04.644 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-19 2020-05-24T14:27:04.645 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:04.645 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:04.646 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:07.414 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:07.415 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:27:12.416 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:12.417 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:12.418 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:15.329 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:15.330 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:27:20.331 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:20.332 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:20.333 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:23.376 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:23.376 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:27:28.378 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:28.379 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:28.380 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:31.428 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:31.429 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:27:36.430 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:36.431 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:36.432 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:39.482 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:39.483 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:27:44.484 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:44.485 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:44.486 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:47.611 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:47.612 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:27:52.613 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:52.614 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:27:52.615 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:27:55.540 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:27:55.540 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:28:00.542 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:00.542 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:00.543 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:03.494 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:03.495 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:28:08.496 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:08.497 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:08.498 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:11.515 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:11.516 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:28:16.517 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:16.518 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:16.519 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:19.561 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:19.562 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:28:24.563 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:24.565 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:24.566 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:27.360 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:27.361 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:28:32.362 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:32.363 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:32.364 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:35.291 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:35.291 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:28:40.292 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:40.293 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:40.294 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:43.274 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:43.275 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:28:48.276 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:48.277 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:48.278 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:51.447 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:51.448 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:28:56.449 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:56.450 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:28:56.450 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:28:59.199 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:28:59.199 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:29:04.200 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:04.201 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:04.202 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:07.003 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:07.004 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:29:12.005 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:12.007 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:12.008 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:14.965 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:14.966 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:29:19.970 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:19.971 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:19.972 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:22.748 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:22.749 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:29:27.750 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:27.750 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:27.751 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:30.706 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:30.707 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:29:35.709 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:35.710 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:35.710 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:38.723 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:38.723 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:29:43.725 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:43.726 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:43.727 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:46.628 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:46.629 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:29:51.631 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:51.632 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:51.633 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:29:54.663 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:29:54.664 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:29:59.666 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:59.667 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:29:59.667 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:02.545 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:02.545 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:30:07.547 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:07.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:07.549 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:10.847 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:10.848 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:30:15.850 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:15.851 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:15.852 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:18.946 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:18.947 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:30:23.949 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:23.950 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:23.950 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:27.206 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:27.206 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:30:32.208 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:32.209 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:32.209 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:35.289 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:35.290 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:30:40.291 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:40.293 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:40.293 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:43.081 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:43.082 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:30:47.308 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:42:21.382 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:30:47.359 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:42:21.432 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:30:47.856 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:42:21.930 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:30:48.083 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:48.084 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:48.085 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:50.796 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:50.796 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:30:55.797 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:55.798 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:30:55.799 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:30:58.910 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:30:58.911 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:31:03.913 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:03.914 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:03.914 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:06.622 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:06.622 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:31:11.624 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:11.626 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:11.627 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:14.480 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:14.480 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:31:19.482 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:19.483 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:19.484 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:22.364 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:22.365 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:31:27.367 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:27.367 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:27.368 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:30.548 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:30.549 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:31:35.552 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:35.553 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:35.554 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:38.379 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:38.380 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:31:43.381 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:43.382 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:43.383 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:46.400 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:46.401 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:31:51.402 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:51.403 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:51.404 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:31:53.367 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:42:20.605 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:31:53.394 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:42:20.633 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:31:53.707 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:42:20.945 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:31:54.682 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:31:54.682 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:31:59.685 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:59.686 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:31:59.687 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:02.858 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:02.859 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:32:07.860 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:07.861 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:07.862 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:10.643 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:10.644 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:32:15.646 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:15.647 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:15.647 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:18.447 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:18.448 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:32:23.449 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:23.450 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:23.451 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:26.690 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:26.691 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:32:31.695 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:31.696 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:31.697 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:34.635 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:34.636 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:32:39.638 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:39.638 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:39.639 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:42.666 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:42.667 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:32:47.668 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:47.670 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:47.671 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:50.797 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:50.798 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:32:55.799 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:55.800 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:32:55.801 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:32:57.044 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:44:31.118 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:32:57.053 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:44:31.128 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:32:57.313 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:44:31.387 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:32:58.616 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:32:58.616 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:33:03.618 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:03.619 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:03.620 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:06.475 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:06.476 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:33:11.478 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:11.479 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:11.479 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:14.397 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:14.398 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:33:19.399 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:19.400 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:19.401 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:22.375 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:22.376 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:33:27.377 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:27.378 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:27.379 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:30.284 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:30.285 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:33:35.286 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:35.288 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:35.289 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:38.590 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:38.591 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:33:43.592 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:43.593 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:43.594 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:46.605 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:46.606 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:33:51.607 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:51.608 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:51.608 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:33:54.513 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:33:54.514 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:33:59.515 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:59.516 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:33:59.517 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:02.269 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:02.270 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:34:07.271 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:07.272 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:07.273 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:08.266 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:44:35.504 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:34:08.273 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:44:35.511 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:34:08.418 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:44:35.656 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:34:10.338 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:10.339 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:34:15.340 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:15.341 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:15.341 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:18.361 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:18.362 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:34:23.364 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:23.365 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:23.365 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:26.183 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:26.184 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:34:31.186 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:31.187 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:31.187 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:34.074 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:34.075 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:34:39.076 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:39.077 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:39.077 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:41.888 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:41.888 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:34:46.890 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:46.891 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:46.892 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:50.077 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:50.078 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:34:55.080 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:55.081 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:34:55.081 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-19 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:34:58.032 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:34:58.032 WARNING:rgw_multi.tests:bucket ajxnhd-19 zone test-zone4 behind zone test-zone2: shard=0 master=00000000005.10.9 target= 2020-05-24T14:35:03.034 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_concurrent_versioned_object_incremental_sync ... FAIL 2020-05-24T14:35:04.107 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:04.108 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:04.108 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:06.182 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T14:35:06.856 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590330903.938053_105.1","last_update":"2020-05-24 14:35:03.938053Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T14:35:06.857 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:35:06.858 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:06.858 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:06.859 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:09.531 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590330903.938053_105.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:35:03.938053Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:35:09.531 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:35:09.532 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:35:09.532 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:09.533 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:09.534 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:12.401 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590330903.938053_105.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:35:03.938053Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:35:12.401 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:35:12.402 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:35:12.403 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:12.403 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:12.404 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:15.410 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590330903.938053_105.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:35:03.938053Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:35:15.411 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:35:16.550 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:16.551 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:16.552 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:19.713 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:35:19.713 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-20 2020-05-24T14:35:19.714 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:19.714 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:19.715 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:22.802 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:35:22.803 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-20 2020-05-24T14:35:22.804 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:22.804 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:22.805 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:25.479 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:35:25.480 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-20 2020-05-24T14:35:25.481 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:25.481 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:25.482 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:28.432 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:35:28.433 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-20 2020-05-24T14:35:28.434 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:28.435 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:28.435 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:31.118 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:35:31.118 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-20 2020-05-24T14:35:31.119 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:31.119 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:31.120 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:34.189 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:35:34.189 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-20 2020-05-24T14:35:34.190 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:34.191 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:34.191 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:36.572 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:35:36.573 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-20 2020-05-24T14:35:36.574 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:36.574 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:36.575 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:39.140 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:35:39.140 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-20 2020-05-24T14:35:39.141 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:39.141 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:39.142 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:42.012 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:35:42.012 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-20 2020-05-24T14:35:42.013 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:42.013 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:42.014 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:44.952 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.2.5"}}}] 2020-05-24T14:35:44.953 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-20 2020-05-24T14:35:44.954 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:44.954 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:44.955 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:47.705 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.2.5"}]} 2020-05-24T14:35:47.706 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-20 2020-05-24T14:35:47.706 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:47.707 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:47.708 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:50.700 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:35:50.700 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:35:55.702 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:55.703 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:35:55.704 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:35:58.728 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:35:58.728 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:03.731 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:03.732 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:03.732 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:07.746 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:07.746 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:12.748 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:12.749 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:12.749 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:15.904 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:15.904 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:20.906 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:20.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:20.908 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:23.876 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:23.877 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:28.879 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:28.880 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:28.881 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:31.900 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:31.901 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:36.902 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:36.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:36.905 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:39.819 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:39.819 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:44.821 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:44.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:44.827 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:47.530 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:47.531 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:36:52.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:52.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:36:52.533 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:36:55.664 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:00.666 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:00.667 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:00.668 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:03.670 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:03.671 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:08.672 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:08.673 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:08.674 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:11.735 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:11.736 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:16.737 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:16.738 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:16.739 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:20.052 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:20.053 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:25.054 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:25.055 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:25.056 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:27.987 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:27.987 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:32.989 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:32.990 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:32.990 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:35.785 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:35.786 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:40.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:40.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:40.789 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:43.725 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:43.726 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:48.728 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:48.728 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:48.729 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:51.895 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:51.895 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:37:56.897 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:56.898 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:37:56.898 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:37:59.732 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:37:59.733 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:04.735 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:04.736 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:04.737 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:07.546 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:07.547 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:12.548 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:12.549 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:12.550 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:15.489 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:15.490 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:20.492 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:20.492 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:20.493 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:23.350 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:23.351 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:28.351 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:28.353 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:28.354 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:31.222 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:31.223 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:36.225 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:36.226 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:36.227 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:39.436 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:39.437 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:44.439 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:44.440 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:44.440 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:47.306 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:47.307 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:38:52.309 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:52.310 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:38:52.311 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:38:55.056 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:38:55.056 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:00.057 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:00.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:00.059 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:02.892 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:02.892 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:07.893 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:07.894 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:07.895 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:08.188 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:50:42.263 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:08.194 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:50:42.267 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:08.563 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:50:42.638 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:11.082 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:11.082 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:16.084 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:16.085 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:16.085 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:18.908 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:18.908 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:23.910 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:23.911 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:23.911 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:26.584 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:26.585 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:31.586 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:31.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:31.588 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:34.883 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:34.884 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:39.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:39.887 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:39.888 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:42.682 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:42.683 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:47.684 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:47.685 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:47.685 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:50.561 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:50.561 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:39:52.518 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:51:26.592 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:52.530 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:51:26.605 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:52.649 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:51:26.723 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:39:55.563 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:55.564 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:39:55.565 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:39:58.664 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:39:58.665 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:03.667 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:03.668 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:03.669 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:06.526 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:06.527 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:11.528 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:11.529 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:11.530 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:14.330 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:50:41.568 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:14.331 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:50:41.569 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:14.552 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:50:41.790 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:40:14.779 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:14.780 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:19.782 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:19.783 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:19.783 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:22.955 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:22.956 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:27.958 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:27.959 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:27.959 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:31.149 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:31.150 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:36.152 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:36.153 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:36.154 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:39.002 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:39.002 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:44.004 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:44.005 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:44.006 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:47.011 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:47.013 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:40:52.014 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:52.014 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:40:52.015 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:40:55.019 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:40:55.020 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:00.020 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:00.021 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:00.022 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:03.013 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:03.013 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:03.794 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:51:31.032 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:41:03.807 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:51:31.043 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:41:03.914 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:51:31.152 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:41:08.015 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:08.016 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:08.017 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:10.725 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:10.725 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:15.727 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:15.728 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:15.729 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:18.811 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:18.812 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:23.813 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:23.814 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:23.815 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:26.800 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:26.801 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:31.803 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:31.803 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:31.804 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:34.710 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:34.711 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:39.712 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:39.713 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:39.713 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:42.835 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:42.836 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:47.838 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:47.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:47.839 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:50.921 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:50.922 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:41:55.924 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:55.925 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:41:55.926 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:41:58.969 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:41:58.970 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:03.972 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:03.973 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:03.974 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:06.982 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:06.983 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:11.984 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:11.985 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:11.986 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:15.415 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:15.416 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:20.418 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:20.419 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:20.419 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:23.760 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:23.761 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:28.763 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:28.764 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:28.764 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:31.790 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:31.791 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:36.793 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:36.794 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:36.795 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:39.760 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:39.761 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:44.763 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:44.764 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:44.765 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:47.666 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:47.666 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:42:52.668 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:52.669 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:42:52.670 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:42:55.755 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:42:55.755 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:00.757 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:00.758 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:00.759 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:03.686 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:03.686 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:08.687 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:08.688 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:08.689 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:11.351 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:11.352 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:16.353 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:16.354 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:16.355 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:19.597 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:19.598 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:24.599 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:24.600 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:24.601 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:27.674 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:27.674 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:32.676 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:32.677 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:32.678 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:35.965 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:35.966 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:40.967 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:40.968 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-20', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:40.969 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-20 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:43.996 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:43:43.996 WARNING:rgw_multi.tests:bucket ajxnhd-20 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.2.5 target= 2020-05-24T14:43:48.998 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_version_suspended_incremental_sync ... FAIL 2020-05-24T14:43:48.999 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-21 2020-05-24T14:43:49.983 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-22 2020-05-24T14:43:51.499 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-23 2020-05-24T14:43:56.515 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:56.516 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:56.517 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:43:58.789 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T14:43:59.522 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590331436.244684_137.1","last_update":"2020-05-24 14:43:56.244684Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T14:43:59.523 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:43:59.524 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:59.524 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:43:59.525 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:01.997 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590331436.244684_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:43:56.244684Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:44:01.998 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:44:01.998 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:44:01.999 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:01.999 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:02.000 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:04.974 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590331436.244684_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:43:56.244684Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:44:04.975 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:44:04.976 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:44:04.976 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:04.977 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:04.978 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:08.269 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590331436.244684_137.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:43:56.244684Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:44:08.270 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:44:17.374 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:17.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:17.376 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:23.220 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000004.8.11"}]} 2020-05-24T14:44:23.221 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-21 2020-05-24T14:44:23.222 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:23.222 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:23.223 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:26.004 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"jwvNUBHil8T-GiJycFT5.PMys.TQf81","ns":""},"count":2},"inc_marker":{"position":"0#00000000004.8.11"}}}] 2020-05-24T14:44:26.006 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-21 2020-05-24T14:44:26.007 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:26.008 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:26.009 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:29.007 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000004.8.11"}]} 2020-05-24T14:44:29.007 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-21 2020-05-24T14:44:29.008 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:29.008 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:29.009 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:31.528 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"jwvNUBHil8T-GiJycFT5.PMys.TQf81","ns":""},"count":2},"inc_marker":{"position":"0#00000000004.8.11"}}}] 2020-05-24T14:44:31.529 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-21 2020-05-24T14:44:31.530 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:31.530 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:31.531 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:35.826 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000004.8.11"}]} 2020-05-24T14:44:35.827 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-21 2020-05-24T14:44:35.827 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:35.828 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:35.828 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:38.898 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"jwvNUBHil8T-GiJycFT5.PMys.TQf81","ns":""},"count":2},"inc_marker":{"position":"0#00000000004.8.11"}}}] 2020-05-24T14:44:38.899 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-21 2020-05-24T14:44:38.899 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:38.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:38.900 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:41.783 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000003.7.5"}]} 2020-05-24T14:44:41.784 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-21 2020-05-24T14:44:41.784 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:41.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:41.785 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:44.576 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"jwvNUBHil8T-GiJycFT5.PMys.TQf81","ns":""},"count":2},"inc_marker":{"position":"0#00000000003.7.5"}}}] 2020-05-24T14:44:44.576 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-21 2020-05-24T14:44:44.577 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:44.577 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:44.578 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:47.322 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000003.7.5"}]} 2020-05-24T14:44:47.323 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-21 2020-05-24T14:44:47.324 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:47.324 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:47.325 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:50.241 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"jwvNUBHil8T-GiJycFT5.PMys.TQf81","ns":""},"count":2},"inc_marker":{"position":"0#00000000003.7.5"}}}] 2020-05-24T14:44:50.241 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-21 2020-05-24T14:44:50.242 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:50.242 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:50.243 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:52.853 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000003.7.5"}]} 2020-05-24T14:44:52.854 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-21 2020-05-24T14:44:52.854 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:52.855 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:44:52.855 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:44:55.794 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:44:55.794 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:45:00.796 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:00.797 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:00.797 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:03.781 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:03.782 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:45:08.784 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:08.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:08.785 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:11.985 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:11.985 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:45:16.987 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:16.989 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:16.990 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:20.018 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:20.018 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:45:25.020 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:25.021 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:25.022 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:28.196 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:28.197 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:45:33.198 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:33.199 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:33.200 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:36.193 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:36.194 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:45:41.195 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:41.196 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:41.197 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:44.183 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:44.184 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:45:49.185 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:49.186 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:49.187 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:45:52.416 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:45:52.417 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:45:57.419 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:57.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:45:57.420 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:00.719 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:00.720 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:46:05.721 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:05.722 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:05.723 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:09.507 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:09.508 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:46:14.509 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:14.511 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:14.512 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:17.742 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:17.743 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:46:22.744 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:22.745 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:22.745 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:25.977 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:25.977 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:46:30.979 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:30.979 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:30.980 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:34.069 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:34.069 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:46:39.076 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:39.077 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:39.077 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:42.460 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:42.460 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:46:47.462 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:47.463 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:47.464 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:49.098 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:58:23.172 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:46:49.117 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:58:23.191 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:46:49.488 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:58:23.562 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:46:50.595 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:50.596 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:46:55.597 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:55.598 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:46:55.599 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:46:58.627 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:46:58.628 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:47:03.629 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:03.630 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:03.631 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:06.559 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:06.559 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:47:11.560 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:11.561 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:11.562 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:14.846 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:14.847 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:47:19.848 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:19.849 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:19.850 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:22.699 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:22.699 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:47:27.700 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:27.701 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:27.702 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:29.285 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:59:03.359 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:29.288 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:59:03.362 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:29.354 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:00:59:03.428 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:47:31.068 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:31.068 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:47:36.070 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:36.071 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:36.071 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:39.080 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:39.081 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:47:44.082 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:44.083 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:44.084 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:46.989 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:46.989 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:47:51.991 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:51.992 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:47:51.993 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:47:55.126 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:47:55.127 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:48:00.128 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:00.129 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:00.129 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:01.190 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:58:28.428 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:01.192 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:58:28.431 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:01.476 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:58:28.714 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:03.145 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:03.146 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:48:08.147 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:08.148 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:08.148 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:11.556 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:11.556 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:48:16.562 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:16.563 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:16.564 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:19.587 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:19.587 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:48:24.588 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:24.590 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:24.590 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:27.634 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:27.635 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:48:32.636 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:32.637 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:32.638 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:35.137 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:59:02.375 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:35.164 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:59:02.402 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:35.192 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:00:59:02.431 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:48:35.767 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:35.768 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:48:40.769 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:40.770 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:40.771 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:43.930 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:43.931 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:48:48.932 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:48.933 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:48.933 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:48:52.325 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:48:52.325 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:48:57.326 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:57.327 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:48:57.328 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:00.862 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:00.862 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:49:05.864 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:05.865 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:05.866 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:08.978 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:08.979 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:49:13.980 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:13.981 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:13.982 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:17.381 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:17.382 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:49:22.384 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:22.385 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:22.394 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:25.229 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:25.230 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:49:30.231 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:30.232 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:30.233 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:33.341 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:33.342 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:49:38.343 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:38.344 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:38.345 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:41.206 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:41.207 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:49:46.208 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:46.209 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:46.210 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:49.312 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:49.313 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:49:54.314 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:54.315 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:49:54.316 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:49:57.380 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:49:57.381 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:50:02.382 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:02.383 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:02.384 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:05.526 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:05.526 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:50:10.528 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:10.529 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:10.530 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:13.545 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:13.546 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:50:18.547 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:18.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:18.549 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:21.357 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:21.357 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:50:26.363 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:26.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:26.365 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:29.348 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:29.348 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:50:34.354 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:34.355 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:34.355 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:37.352 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:37.353 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:50:42.355 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:42.356 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:42.356 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:45.169 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:45.170 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:50:50.178 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:50.179 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:50.180 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:50:53.509 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:50:53.509 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:50:58.514 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:58.515 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:50:58.515 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:01.594 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:01.594 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:51:06.595 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:06.596 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:06.597 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:09.896 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:09.897 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:51:14.898 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:14.899 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:14.900 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:18.133 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:18.134 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:51:23.135 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:23.136 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:23.137 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:26.455 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:26.456 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:51:31.458 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:31.458 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:31.459 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:34.967 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:34.967 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:51:39.969 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:39.970 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:39.970 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:43.044 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:43.045 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:51:48.046 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:48.047 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:48.047 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:51:51.561 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:51:51.561 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:51:56.563 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:56.564 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:51:56.564 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:00.166 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:00.167 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:52:05.169 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:05.170 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:05.170 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:08.653 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:08.653 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:52:13.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:13.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:13.656 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:17.018 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:17.018 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:52:22.019 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:22.020 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:22.021 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:24.807 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:24.807 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:52:29.808 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:29.809 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:29.811 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:32.933 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:32.934 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:52:37.935 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:37.936 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:37.937 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:41.186 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:41.187 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:52:46.188 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:46.189 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:46.190 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:49.302 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:49.303 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:52:54.304 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:54.305 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-21', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:52:54.306 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-21 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:52:57.648 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:52:57.649 WARNING:rgw_multi.tests:bucket ajxnhd-21 zone test-zone4 behind zone test-zone2: shard=0 master=00000000003.7.5 target= 2020-05-24T14:53:02.651 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_delete_marker_full_sync ... FAIL 2020-05-24T14:53:02.652 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-24 2020-05-24T14:53:03.584 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-25 2020-05-24T14:53:05.087 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-26 2020-05-24T14:53:12.790 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:12.791 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:12.792 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:14.474 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T14:53:15.169 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590331992.462010_167.1","last_update":"2020-05-24 14:53:12.462010Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T14:53:15.169 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T14:53:15.170 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:15.170 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:15.171 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:17.744 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590331992.462010_167.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:53:12.462010Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:53:17.745 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T14:53:17.746 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T14:53:17.747 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:17.747 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:17.748 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:20.410 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590331992.462010_167.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:53:12.462010Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:53:20.410 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T14:53:20.411 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T14:53:20.411 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:20.412 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:20.412 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:23.418 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590331992.462010_167.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 14:53:12.462010Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T14:53:23.418 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T14:53:28.790 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:28.791 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:28.791 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:34.831 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000005.20.6"}]} 2020-05-24T14:53:34.832 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-24 2020-05-24T14:53:34.832 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:34.833 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:34.833 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:37.346 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000005.20.6"}}}] 2020-05-24T14:53:37.347 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-24 2020-05-24T14:53:37.347 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:37.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:37.349 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:40.328 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000005.20.6"}]} 2020-05-24T14:53:40.329 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-24 2020-05-24T14:53:40.330 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:40.330 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:40.331 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:43.040 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000005.20.6"}}}] 2020-05-24T14:53:43.041 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-24 2020-05-24T14:53:43.042 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:43.042 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:43.043 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:45.332 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:05:19.406 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:53:45.334 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:05:19.408 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:53:45.657 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:05:19.732 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:53:45.856 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000005.20.6"}]} 2020-05-24T14:53:45.857 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-24 2020-05-24T14:53:45.857 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:45.858 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:45.858 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:49.321 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000005.20.6"}}}] 2020-05-24T14:53:49.322 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-24 2020-05-24T14:53:49.322 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:49.323 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:49.324 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:51.990 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000001.10.12"}]} 2020-05-24T14:53:51.990 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-24 2020-05-24T14:53:51.991 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:51.992 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:51.992 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:54.675 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000001.10.12"}}}] 2020-05-24T14:53:54.676 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-24 2020-05-24T14:53:54.677 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:54.678 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:54.679 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:53:57.193 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000001.10.12"}]} 2020-05-24T14:53:57.194 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-24 2020-05-24T14:53:57.194 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:57.195 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:53:57.196 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:00.180 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj","instance":"null","ns":""},"count":1},"inc_marker":{"position":"0#00000000001.10.12"}}}] 2020-05-24T14:54:00.180 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-24 2020-05-24T14:54:00.181 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:00.182 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:00.182 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:02.584 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000001.10.12"}]} 2020-05-24T14:54:02.584 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-24 2020-05-24T14:54:02.585 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:02.585 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:02.586 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:05.505 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:05.506 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:54:10.507 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:10.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:10.508 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:13.584 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:13.584 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:54:18.586 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:18.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:18.587 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:21.864 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:21.864 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:54:26.865 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:26.866 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:26.867 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:30.291 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:30.292 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:54:35.293 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:35.295 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:35.296 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:38.854 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:38.854 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:54:43.856 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:43.856 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:43.857 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:47.082 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:47.083 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:54:52.084 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:52.085 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:54:52.085 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:54:55.082 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:54:55.083 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:54:55.791 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:05:23.029 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:54:55.808 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:05:23.046 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:54:56.237 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:05:23.476 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:55:00.084 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:00.085 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:00.085 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:03.416 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:03.416 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:55:08.418 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:08.418 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:08.419 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:11.722 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:11.723 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:55:16.724 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:16.725 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:16.726 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:20.129 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:20.130 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:55:25.131 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:25.132 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:25.133 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:28.061 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:28.062 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:55:33.063 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:33.064 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:33.065 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:36.335 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:36.375 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:55:41.376 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:41.377 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:41.378 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:44.527 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:44.527 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:55:49.529 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:49.530 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:49.530 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:55:50.072 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:07:24.146 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:55:50.077 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:07:24.152 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:55:50.149 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:07:24.223 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T14:55:54.082 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:55:54.083 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:55:59.084 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:59.085 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:55:59.086 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:03.301 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:03.302 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:56:08.303 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:08.304 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:08.305 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:12.395 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:12.396 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:56:17.398 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:17.399 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:17.399 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:20.428 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:20.428 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:56:25.434 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:25.435 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:25.436 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:28.622 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:28.622 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:56:33.624 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:33.625 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:33.625 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:36.963 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:36.963 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:56:41.965 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:41.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:41.966 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:45.165 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:45.166 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:56:50.167 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:50.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:50.169 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:56:53.393 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:56:53.393 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:56:56.145 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:07:23.384 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:56:56.159 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:07:23.398 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:56:56.331 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:07:23.570 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T14:56:58.395 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:58.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:56:58.397 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:01.785 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:01.786 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:57:06.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:06.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:06.788 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:09.972 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:09.973 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:57:14.974 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:14.975 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:14.976 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:18.343 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:18.344 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:57:23.346 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:23.347 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:23.347 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:26.519 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:26.520 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:57:31.521 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:31.522 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:31.523 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:34.791 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:34.792 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:57:39.793 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:39.794 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:39.795 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:43.120 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:43.120 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:57:48.122 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:48.123 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:48.123 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:57:51.534 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:57:51.534 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:57:56.536 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:56.537 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:57:56.538 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:00.084 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:00.085 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:58:05.086 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:05.087 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:05.088 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:08.150 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:08.151 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:58:13.152 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:13.153 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:13.154 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:16.629 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:16.630 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:58:21.632 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:21.633 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:21.634 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:25.355 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:25.356 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:58:30.357 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:30.358 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:30.358 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:33.603 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:33.603 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:58:38.604 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:38.605 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:38.606 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:41.709 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:41.710 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:58:46.711 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:46.712 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:46.713 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:49.820 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:49.821 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:58:54.822 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:54.824 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:58:54.825 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:58:58.097 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:58:58.098 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:59:03.099 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:03.100 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:03.101 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:06.494 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:06.495 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:59:11.496 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:11.497 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:11.498 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:15.122 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:15.123 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:59:20.124 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:20.125 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:20.126 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:23.528 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:23.529 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:59:28.531 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:28.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:28.533 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:31.906 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:31.907 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:59:36.911 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:36.912 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:36.913 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:40.144 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:40.145 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:59:45.146 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:45.147 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:45.148 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:48.324 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:48.325 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T14:59:53.326 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:53.327 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T14:59:53.328 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T14:59:56.534 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T14:59:56.535 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:00:01.536 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:01.537 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:01.538 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:05.050 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:05.051 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:00:10.052 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:10.053 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:10.054 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:13.257 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:13.258 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:00:18.259 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:18.260 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:18.261 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:21.375 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:21.376 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:00:26.377 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:26.378 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:26.379 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:29.523 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:29.526 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:00:34.527 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:34.528 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:34.529 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:37.846 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:37.847 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:00:39.945 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:12:14.019 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:00:39.951 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:12:14.026 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:00:40.206 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:12:14.281 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:00:42.849 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:42.850 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:42.850 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:45.888 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:45.888 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:00:50.890 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:50.891 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:50.892 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:00:54.169 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:00:54.170 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:00:59.171 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:59.172 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:00:59.174 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:02.453 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:02.454 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:01:07.455 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:07.456 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:07.457 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:10.563 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:10.564 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:01:15.565 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:15.566 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:15.567 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:18.867 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:18.868 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:01:23.869 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:23.871 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:23.872 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:27.309 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:27.310 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:01:32.312 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:32.313 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:32.313 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:35.508 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:35.508 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:01:40.510 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:40.511 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:40.511 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:43.852 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:43.853 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:01:48.855 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:48.856 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:48.856 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:01:52.186 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:01:52.186 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:01:52.325 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:12:19.563 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:01:52.358 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:12:19.596 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:01:52.738 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:12:19.976 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:01:57.187 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:57.188 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:01:57.189 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:00.680 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:00.681 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:02:05.682 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:05.683 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:05.684 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:08.826 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:08.827 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:02:13.828 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:13.829 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-24', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:13.830 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-24 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:17.429 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:02:17.430 WARNING:rgw_multi.tests:bucket ajxnhd-24 zone test-zone4 behind zone test-zone2: shard=0 master=00000000001.10.12 target= 2020-05-24T15:02:22.432 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_suspended_delete_marker_full_sync ... FAIL 2020-05-24T15:02:22.433 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-27 2020-05-24T15:02:23.336 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-28 2020-05-24T15:02:24.823 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-29 2020-05-24T15:02:29.386 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_versioning ... ok 2020-05-24T15:02:29.387 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-30 2020-05-24T15:02:30.664 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-31 2020-05-24T15:02:32.742 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-32 2020-05-24T15:02:38.238 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_acl ... ok 2020-05-24T15:02:38.239 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-33 2020-05-24T15:02:39.833 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-34 2020-05-24T15:02:42.177 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-35 2020-05-24T15:02:49.134 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_cors ... ok 2020-05-24T15:02:49.135 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-36 2020-05-24T15:02:50.652 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-37 2020-05-24T15:02:52.162 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-38 2020-05-24T15:02:54.113 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:54.114 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:54.114 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:56.336 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T15:02:56.918 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332573.500710_242.1","last_update":"2020-05-24 15:02:53.500710Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T15:02:56.919 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:02:56.920 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:56.920 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:56.921 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:02:59.809 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332573.500710_242.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:02:53.500710Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:02:59.810 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:02:59.810 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:02:59.811 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:59.811 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:02:59.812 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:02.744 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332573.500710_242.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:02:53.500710Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:02.745 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:03:02.745 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:03:02.746 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:02.746 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:02.747 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:05.846 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332573.500710_242.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:02:53.500710Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:05.846 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:03:11.530 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_delete_notempty ... ok 2020-05-24T15:03:11.531 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-39 2020-05-24T15:03:14.658 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-40 2020-05-24T15:03:17.039 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-41 2020-05-24T15:03:18.859 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:18.860 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:18.860 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:20.983 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=114a3cfc-32a6-4ae9-bab5-baac4f59a79e 2020-05-24T15:03:21.652 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"1_1590328568.276896_39.1","last_update":"2020-05-24 13:56:08.276896Z"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332598.292976_254.1","last_update":"2020-05-24 15:03:18.292976Z"},{"marker":"1_1590328767.216616_146.1","last_update":"2020-05-24 13:59:27.216616Z"}] 2020-05-24T15:03:21.652 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:03:21.653 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:21.654 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:21.654 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:24.466 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332598.292976_254.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:18.292976Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:24.467 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:03:24.467 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:03:24.468 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:24.468 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:24.469 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:27.331 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332598.292976_254.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:18.292976Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:27.332 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:03:27.332 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:03:27.333 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:27.333 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:03:27.334 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:03:30.509 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","realm_epoch":2},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332598.292976_254.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:03:18.292976Z","realm_epoch":2}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:03:30.510 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:03:30.511 DEBUG:tasks.rgw.c1.client.1:waiting for process to exit 2020-05-24T15:03:30.511 INFO:teuthology.orchestra.run:waiting for 300 2020-05-24T15:03:30.530 INFO:tasks.rgw.c1.client.1.smithi050.stdout:2020-05-24 15:03:30.516 1b865700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends civetweb port=8001 -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 16544) UID: 0 2020-05-24T15:03:30.530 INFO:tasks.rgw.c1.client.1.smithi050.stdout:2020-05-24 15:03:30.521 4049040 -1 shutting down 2020-05-24T15:03:36.614 INFO:tasks.rgw.c1.client.1:Stopped 2020-05-24T15:03:36.615 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:03:36.615 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:03:36.616 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --master --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:03:37.118 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:03:37.114 7f42965a1840 0 NOTICE: overriding master zone: d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T15:03:37.148 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:03:37.144 7f42965a1840 0 failed reading obj info from .rgw.root:zone_info.88888d62-7b54-4b85-bd95-327afc14fb31: (2) No such file or directory 2020-05-24T15:03:37.148 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:03:37.144 7f42965a1840 0 WARNING: could not read zone params for zone id=88888d62-7b54-4b85-bd95-327afc14fb31 name=test-zone3 2020-05-24T15:03:37.183 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:03:37.179 7f42965a1840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T15:03:37.184 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:03:37.179 7f42965a1840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T15:03:37.370 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:03:37.371 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:03:37.372 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:03:37.372 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:03:42.079 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","epoch":1,"predecessor_uuid":"114a3cfc-32a6-4ae9-bab5-baac4f59a79e","sync_status":["1_1590328568.276896_39.1","","1_1590332598.292976_254.1","1_1590328767.216616_146.1"],"period_map":{"id":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"8de99cfe-4678-427d-bca8-2f77415e7812","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"8de99cfe-4678-427d-bca8-2f77415e7812","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":3} 2020-05-24T15:03:42.080 INFO:rgw_multi.tests:Set master zone=test-zone2, waiting 60s for reconfiguration.. 2020-05-24T15:04:10.884 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:15:44.959 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:04:10.887 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:15:44.962 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:04:10.994 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:15:45.069 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:04:42.081 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-42 2020-05-24T15:04:43.616 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-43 2020-05-24T15:04:44.279 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:04:44.280 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:04:44.281 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:04:46.395 INFO:teuthology.orchestra.run.smithi160.stderr:No --period given, using current period=7920f6a5-aa4e-4bd1-aca6-86793c8b287f 2020-05-24T15:04:47.211 INFO:teuthology.orchestra.run.smithi160.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332684.147703_160.1","last_update":"2020-05-24 15:04:44.147703Z"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:04:47.212 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone1 2020-05-24T15:04:47.213 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:04:47.214 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:04:47.215 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:04:49.880 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","realm_epoch":3},"markers":[{"key":0,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":14,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":3}},{"key":2,"val":{"state":1,"marker":"1_1590332684.147703_160.1","next_step_marker":"","total_entries":55,"pos":0,"timestamp":"2020-05-24 15:04:44.147703Z","realm_epoch":3}},{"key":3,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":10,"pos":0,"timestamp":"0.000000","realm_epoch":3}}]},"full_sync":{"total":80,"complete":80}} 2020-05-24T15:04:49.881 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone1 2020-05-24T15:04:49.882 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:04:49.883 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'modify', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:04:49.883 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone modify --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --master --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:04:50.402 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:04:50.401 7f64b1cbb840 0 NOTICE: overriding master zone: 8de99cfe-4678-427d-bca8-2f77415e7812 2020-05-24T15:04:50.419 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:04:50.418 7f64b1cbb840 0 failed reading obj info from .rgw.root:zone_info.1a48d5d2-db65-4e7e-9296-4bd2463c8a06: (2) No such file or directory 2020-05-24T15:04:50.419 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:04:50.418 7f64b1cbb840 0 WARNING: could not read zone params for zone id=1a48d5d2-db65-4e7e-9296-4bd2463c8a06 name=test-zone4 2020-05-24T15:04:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:04:50.460 7f64b1cbb840 0 failed reading obj info from .rgw.root:zone_info.8de99cfe-4678-427d-bca8-2f77415e7812: (2) No such file or directory 2020-05-24T15:04:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:04:50.460 7f64b1cbb840 0 WARNING: could not read zone params for zone id=8de99cfe-4678-427d-bca8-2f77415e7812 name=test-zone2 2020-05-24T15:04:50.663 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","domain_root":"test-zone1.rgw.meta:root","control_pool":"test-zone1.rgw.control","gc_pool":"test-zone1.rgw.log:gc","lc_pool":"test-zone1.rgw.log:lc","log_pool":"test-zone1.rgw.log","intent_log_pool":"test-zone1.rgw.log:intent","usage_log_pool":"test-zone1.rgw.log:usage","reshard_pool":"test-zone1.rgw.log:reshard","user_keys_pool":"test-zone1.rgw.meta:users.keys","user_email_pool":"test-zone1.rgw.meta:users.email","user_swift_pool":"test-zone1.rgw.meta:users.swift","user_uid_pool":"test-zone1.rgw.meta:users.uid","otp_pool":"test-zone1.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone1.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone1.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone1.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:04:50.664 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:04:50.665 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:04:50.666 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:04:55.514 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":1,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:04:55.515 INFO:rgw_multi.tests:Set master zone=test-zone1, waiting 60s for reconfiguration.. 2020-05-24T15:05:16.830 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:15:44.068 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:05:16.831 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:15:44.069 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:05:16.931 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:15:44.169 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:05:55.516 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-44 2020-05-24T15:05:56.448 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-45 2020-05-24T15:05:57.981 INFO:tasks.rgw.c1.client.1:Restarting daemon 2020-05-24T15:05:57.982 INFO:teuthology.orchestra.run.smithi050:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/c1.client.1.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2020-05-24T15:05:57.987 INFO:tasks.rgw.c1.client.1:Started 2020-05-24T15:05:57.987 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T15:05:58.089 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:05:58.089 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:05:58.091 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8001; Connection refused 2020-05-24T15:05:58.091 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T15:05:59.092 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T15:05:59.178 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:05:59.180 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:05:59.182 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8001; Connection refused 2020-05-24T15:05:59.183 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T15:06:01.184 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T15:06:01.239 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:06:01.239 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:06:01.243 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8001; Connection refused 2020-05-24T15:06:01.244 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T15:06:05.245 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T15:06:05.309 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:06:05.309 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:06:05.313 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi050.front.sepia.ceph.com:8001; Connection refused 2020-05-24T15:06:05.314 DEBUG:teuthology.orchestra.run:got remote process result: 7 2020-05-24T15:06:13.315 INFO:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:8001 2020-05-24T15:06:13.363 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2020-05-24T15:06:13.364 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2020-05-24T15:06:13.469 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 2012 0 --:--:-- --:--:-- --:--:-- 2018 2020-05-24T15:06:13.469 INFO:teuthology.orchestra.run.smithi050.stdout:anonymous 2020-05-24T15:06:13.470 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:13.471 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:13.471 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:16.771 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=f35208d4-7215-4668-86dd-a246cde34636 2020-05-24T15:06:17.789 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590332757.336629_109.1","last_update":"2020-05-24 15:05:57.336629Z"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:06:17.790 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:06:17.791 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:17.792 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:17.793 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:20.571 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332757.336629_109.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:05:57.336629Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:06:20.572 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:06:20.573 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:06:20.573 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:20.574 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:20.574 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:23.471 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332757.336629_109.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:05:57.336629Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:06:23.472 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:06:23.472 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:06:23.473 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:23.473 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:23.474 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:27.036 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332757.336629_109.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:05:57.336629Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:06:27.037 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:06:27.037 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-39 zones={test-zone2, test-zone1} 2020-05-24T15:06:28.342 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:28.456 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:28.642 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-39 zones={test-zone2, test-zone1} 2020-05-24T15:06:28.643 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-39 zones={test-zone3, test-zone1} 2020-05-24T15:06:29.088 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:29.279 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:29.438 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-39 zones={test-zone3, test-zone1} 2020-05-24T15:06:29.439 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-39 zones={test-zone1, test-zone2} 2020-05-24T15:06:29.562 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:29.592 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:29.834 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-39 zones={test-zone1, test-zone2} 2020-05-24T15:06:29.835 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-39 zones={test-zone3, test-zone2} 2020-05-24T15:06:29.947 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:30.002 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:30.170 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-39 zones={test-zone3, test-zone2} 2020-05-24T15:06:30.171 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-39 zones={test-zone1, test-zone3} 2020-05-24T15:06:30.254 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:30.350 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:30.535 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-39 zones={test-zone1, test-zone3} 2020-05-24T15:06:30.536 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-39 zones={test-zone2, test-zone3} 2020-05-24T15:06:30.623 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:30.718 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:30.847 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-39 zones={test-zone2, test-zone3} 2020-05-24T15:06:30.848 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-40 zones={test-zone2, test-zone1} 2020-05-24T15:06:31.386 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:31.468 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:31.634 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-40 zones={test-zone2, test-zone1} 2020-05-24T15:06:31.635 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-40 zones={test-zone3, test-zone1} 2020-05-24T15:06:31.868 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:31.966 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:32.166 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-40 zones={test-zone3, test-zone1} 2020-05-24T15:06:32.167 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-40 zones={test-zone1, test-zone2} 2020-05-24T15:06:32.297 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:32.333 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:32.478 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-40 zones={test-zone1, test-zone2} 2020-05-24T15:06:32.479 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-40 zones={test-zone3, test-zone2} 2020-05-24T15:06:32.533 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:32.610 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:32.744 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-40 zones={test-zone3, test-zone2} 2020-05-24T15:06:32.745 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-40 zones={test-zone1, test-zone3} 2020-05-24T15:06:32.819 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:32.930 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:33.110 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-40 zones={test-zone1, test-zone3} 2020-05-24T15:06:33.111 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-40 zones={test-zone2, test-zone3} 2020-05-24T15:06:33.201 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:33.294 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:33.455 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-40 zones={test-zone2, test-zone3} 2020-05-24T15:06:33.456 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-41 zones={test-zone2, test-zone1} 2020-05-24T15:06:33.838 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:33.929 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:34.142 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-41 zones={test-zone2, test-zone1} 2020-05-24T15:06:34.143 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-41 zones={test-zone3, test-zone1} 2020-05-24T15:06:34.454 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:34.507 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:34.746 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-41 zones={test-zone3, test-zone1} 2020-05-24T15:06:34.747 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-41 zones={test-zone1, test-zone2} 2020-05-24T15:06:34.875 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:34.947 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:35.138 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-41 zones={test-zone1, test-zone2} 2020-05-24T15:06:35.139 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-41 zones={test-zone3, test-zone2} 2020-05-24T15:06:35.199 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:35.275 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:35.421 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-41 zones={test-zone3, test-zone2} 2020-05-24T15:06:35.422 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-41 zones={test-zone1, test-zone3} 2020-05-24T15:06:35.509 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:35.568 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:35.718 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-41 zones={test-zone1, test-zone3} 2020-05-24T15:06:35.719 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-41 zones={test-zone2, test-zone3} 2020-05-24T15:06:35.798 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:35.845 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:35.977 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-41 zones={test-zone2, test-zone3} 2020-05-24T15:06:35.978 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-42 zones={test-zone2, test-zone1} 2020-05-24T15:06:36.083 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:36.114 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:36.310 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-42 zones={test-zone2, test-zone1} 2020-05-24T15:06:36.311 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-42 zones={test-zone3, test-zone1} 2020-05-24T15:06:36.400 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:36.428 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:36.702 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-42 zones={test-zone3, test-zone1} 2020-05-24T15:06:36.703 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-42 zones={test-zone1, test-zone2} 2020-05-24T15:06:36.817 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:36.859 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:37.051 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-42 zones={test-zone1, test-zone2} 2020-05-24T15:06:37.052 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-42 zones={test-zone3, test-zone2} 2020-05-24T15:06:37.165 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:37.196 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:37.407 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-42 zones={test-zone3, test-zone2} 2020-05-24T15:06:37.407 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-42 zones={test-zone1, test-zone3} 2020-05-24T15:06:37.468 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:37.568 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:37.700 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-42 zones={test-zone1, test-zone3} 2020-05-24T15:06:37.701 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-42 zones={test-zone2, test-zone3} 2020-05-24T15:06:37.792 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:37.906 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:38.092 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-42 zones={test-zone2, test-zone3} 2020-05-24T15:06:38.094 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-43 zones={test-zone2, test-zone1} 2020-05-24T15:06:38.203 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:38.243 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:38.385 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-43 zones={test-zone2, test-zone1} 2020-05-24T15:06:38.385 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-43 zones={test-zone3, test-zone1} 2020-05-24T15:06:38.447 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:38.538 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:38.685 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-43 zones={test-zone3, test-zone1} 2020-05-24T15:06:38.686 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-43 zones={test-zone1, test-zone2} 2020-05-24T15:06:38.773 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:38.804 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:39.062 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-43 zones={test-zone1, test-zone2} 2020-05-24T15:06:39.063 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-43 zones={test-zone3, test-zone2} 2020-05-24T15:06:39.131 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:39.226 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:39.426 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-43 zones={test-zone3, test-zone2} 2020-05-24T15:06:39.427 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-43 zones={test-zone1, test-zone3} 2020-05-24T15:06:39.487 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:39.558 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:39.689 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-43 zones={test-zone1, test-zone3} 2020-05-24T15:06:39.690 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-43 zones={test-zone2, test-zone3} 2020-05-24T15:06:39.770 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:39.862 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:40.023 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-43 zones={test-zone2, test-zone3} 2020-05-24T15:06:40.024 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-44 zones={test-zone2, test-zone1} 2020-05-24T15:06:40.092 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:40.131 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:40.261 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-44 zones={test-zone2, test-zone1} 2020-05-24T15:06:40.262 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-44 zones={test-zone3, test-zone1} 2020-05-24T15:06:40.353 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:40.410 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:40.561 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-44 zones={test-zone3, test-zone1} 2020-05-24T15:06:40.562 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-44 zones={test-zone1, test-zone2} 2020-05-24T15:06:40.705 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:40.774 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:40.938 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-44 zones={test-zone1, test-zone2} 2020-05-24T15:06:40.939 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-44 zones={test-zone3, test-zone2} 2020-05-24T15:06:40.992 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:41.070 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:41.201 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-44 zones={test-zone3, test-zone2} 2020-05-24T15:06:41.202 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-44 zones={test-zone1, test-zone3} 2020-05-24T15:06:41.278 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:41.320 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:41.485 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-44 zones={test-zone1, test-zone3} 2020-05-24T15:06:41.486 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-44 zones={test-zone2, test-zone3} 2020-05-24T15:06:41.567 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:41.690 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:41.817 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-44 zones={test-zone2, test-zone3} 2020-05-24T15:06:41.818 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-45 zones={test-zone2, test-zone1} 2020-05-24T15:06:41.890 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:41.928 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:42.061 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-45 zones={test-zone2, test-zone1} 2020-05-24T15:06:42.062 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-45 zones={test-zone3, test-zone1} 2020-05-24T15:06:42.179 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:42.250 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:42.419 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-45 zones={test-zone3, test-zone1} 2020-05-24T15:06:42.420 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-45 zones={test-zone1, test-zone2} 2020-05-24T15:06:42.526 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:42.559 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:42.766 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-45 zones={test-zone1, test-zone2} 2020-05-24T15:06:42.767 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-45 zones={test-zone3, test-zone2} 2020-05-24T15:06:42.878 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:42.970 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:43.178 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-45 zones={test-zone3, test-zone2} 2020-05-24T15:06:43.179 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-45 zones={test-zone1, test-zone3} 2020-05-24T15:06:43.296 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:43.348 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:43.574 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-45 zones={test-zone1, test-zone3} 2020-05-24T15:06:43.575 INFO:tasks.rgw_multi.zone_rados:comparing bucket=ajxnhd-45 zones={test-zone2, test-zone3} 2020-05-24T15:06:43.682 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2020-05-24T15:06:43.734 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2020-05-24T15:06:43.912 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=ajxnhd-45 zones={test-zone2, test-zone3} 2020-05-24T15:06:43.912 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:43.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:43.914 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 114a3cfc-32a6-4ae9-bab5-baac4f59a79e --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:46.765 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":"1_1590328567.776739_38.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:07.776739104Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ui15Zsog5OeOBfQLFnhWmAY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328568.276896_39.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:08.276896545Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ui15Zsog5OeOBfQLFnhWmAY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328780.710920_36.1","section":"bucket.instance","name":"ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","timestamp":"2020-05-24 13:59:40.710920367Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_1x1ZZjoXeMNzJoUjkg5bWmM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328780.907463_37.1","section":"bucket.instance","name":"ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","timestamp":"2020-05-24 13:59:40.907463503Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_1x1ZZjoXeMNzJoUjkg5bWmM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328780.992516_38.1","section":"bucket","name":"ajxnhd-10","timestamp":"2020-05-24 13:59:40.992516918Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_F0eV8p4gy4VVoOyBszXK9Uy","ver":1},"status":{"status":"write"}}},{"id":"1_1590328781.173823_39.1","section":"bucket","name":"ajxnhd-10","timestamp":"2020-05-24 13:59:41.173823390Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_F0eV8p4gy4VVoOyBszXK9Uy","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328781.786763_40.1","section":"bucket.instance","name":"ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","timestamp":"2020-05-24 13:59:41.786763813Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yAfrDCGApzmfUfpLV32eRYD","ver":1},"status":{"status":"write"}}},{"id":"1_1590328782.049055_41.1","section":"bucket.instance","name":"ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","timestamp":"2020-05-24 13:59:42.049055331Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yAfrDCGApzmfUfpLV32eRYD","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328782.155023_42.1","section":"bucket","name":"ajxnhd-11","timestamp":"2020-05-24 13:59:42.155023264Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6JZ7TwZSk3ER0WTRkMdpYxK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328782.443391_43.1","section":"bucket","name":"ajxnhd-11","timestamp":"2020-05-24 13:59:42.443391351Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6JZ7TwZSk3ER0WTRkMdpYxK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.411331_44.1","section":"bucket.instance","name":"ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","timestamp":"2020-05-24 13:59:43.411331039Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5HqOnT066cNskmDRsRqZ-dE","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.597981_45.1","section":"bucket.instance","name":"ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","timestamp":"2020-05-24 13:59:43.597981263Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5HqOnT066cNskmDRsRqZ-dE","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.683466_46.1","section":"bucket","name":"ajxnhd-12","timestamp":"2020-05-24 13:59:43.683466272Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_z0ez1Ljq8OH081qJv8Q7CL6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.837755_47.1","section":"bucket","name":"ajxnhd-12","timestamp":"2020-05-24 13:59:43.837755756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_z0ez1Ljq8OH081qJv8Q7CL6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329320.125728_54.1","section":"bucket.instance","name":"ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","timestamp":"2020-05-24 14:08:40.125728151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NEqYrxBauZKMtGHHXFkIZCq","ver":1},"status":{"status":"write"}}},{"id":"1_1590329320.290517_55.1","section":"bucket.instance","name":"ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","timestamp":"2020-05-24 14:08:40.290517017Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NEqYrxBauZKMtGHHXFkIZCq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329320.400997_56.1","section":"bucket","name":"ajxnhd-13","timestamp":"2020-05-24 14:08:40.400997847Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eweXCos3PdlUyYekfNFjSHH","ver":1},"status":{"status":"write"}}},{"id":"1_1590329320.572773_57.1","section":"bucket","name":"ajxnhd-13","timestamp":"2020-05-24 14:08:40.572773700Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eweXCos3PdlUyYekfNFjSHH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329322.157703_58.1","section":"bucket.instance","name":"ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","timestamp":"2020-05-24 14:08:42.157703103Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BmyQDJAOboCFiJ45lLtofKE","ver":1},"status":{"status":"write"}}},{"id":"1_1590329322.303444_59.1","section":"bucket.instance","name":"ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","timestamp":"2020-05-24 14:08:42.303444730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BmyQDJAOboCFiJ45lLtofKE","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329322.485950_60.1","section":"bucket","name":"ajxnhd-14","timestamp":"2020-05-24 14:08:42.485950917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_quzz_j7V4et7RpGVfsZ8DI3","ver":1},"status":{"status":"write"}}},{"id":"1_1590329322.753097_61.1","section":"bucket","name":"ajxnhd-14","timestamp":"2020-05-24 14:08:42.753097194Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_quzz_j7V4et7RpGVfsZ8DI3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329323.912546_62.1","section":"bucket.instance","name":"ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","timestamp":"2020-05-24 14:08:43.912546421Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ljytbx-c-ioF_jOyI19KhIn","ver":1},"status":{"status":"write"}}},{"id":"1_1590329324.055027_63.1","section":"bucket.instance","name":"ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","timestamp":"2020-05-24 14:08:44.055027596Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ljytbx-c-ioF_jOyI19KhIn","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329324.140936_64.1","section":"bucket","name":"ajxnhd-15","timestamp":"2020-05-24 14:08:44.140936259Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__NTNWGbez_pXSCfSDkXBqXJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590329324.279876_65.1","section":"bucket","name":"ajxnhd-15","timestamp":"2020-05-24 14:08:44.279876745Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__NTNWGbez_pXSCfSDkXBqXJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329851.002187_72.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:31.002187986Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"status":{"status":"write"}}},{"id":"1_1590329851.192404_73.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:31.192404499Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329851.255970_74.1","section":"bucket","name":"ajxnhd-16","timestamp":"2020-05-24 14:17:31.255970913Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_n5pa9q08aZnglX0ZBX2MI72","ver":1},"status":{"status":"write"}}},{"id":"1_1590329851.450578_75.1","section":"bucket","name":"ajxnhd-16","timestamp":"2020-05-24 14:17:31.450578151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_n5pa9q08aZnglX0ZBX2MI72","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329852.156682_76.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:32.156682053Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"status":{"status":"write"}}},{"id":"1_1590329852.314606_77.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:32.314606785Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329852.385946_78.1","section":"bucket","name":"ajxnhd-17","timestamp":"2020-05-24 14:17:32.385946764Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2yVMkwTR0dwxoaxsJDfspmT","ver":1},"status":{"status":"write"}}},{"id":"1_1590329852.537860_79.1","section":"bucket","name":"ajxnhd-17","timestamp":"2020-05-24 14:17:32.537860267Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2yVMkwTR0dwxoaxsJDfspmT","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329853.921024_80.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:33.921024364Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"status":{"status":"write"}}},{"id":"1_1590329854.087205_81.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:34.087205756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329854.204871_82.1","section":"bucket","name":"ajxnhd-18","timestamp":"2020-05-24 14:17:34.204871979Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBs2gR836NQmjWKNfHbjG8K","ver":1},"status":{"status":"write"}}},{"id":"1_1590329854.445256_83.1","section":"bucket","name":"ajxnhd-18","timestamp":"2020-05-24 14:17:34.445256737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBs2gR836NQmjWKNfHbjG8K","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329855.079839_84.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:35.079839696Z","data":{"read_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":2},"status":{"status":"write"}}},{"id":"1_1590329855.330800_85.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:35.330800723Z","data":{"read_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329856.118614_86.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:36.118614343Z","data":{"read_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":2},"status":{"status":"write"}}},{"id":"1_1590329856.475914_87.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:36.475914391Z","data":{"read_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329857.623824_88.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:37.623824536Z","data":{"read_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":2},"status":{"status":"write"}}},{"id":"1_1590329857.827328_89.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:37.827328629Z","data":{"read_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330379.734826_96.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:19.734826079Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"status":{"status":"write"}}},{"id":"1_1590330379.889098_97.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:19.889098852Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330380.055101_98.1","section":"bucket","name":"ajxnhd-19","timestamp":"2020-05-24 14:26:20.055101601Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VogH8d_v03-pKxveKvcQRpR","ver":1},"status":{"status":"write"}}},{"id":"1_1590330380.186271_99.1","section":"bucket","name":"ajxnhd-19","timestamp":"2020-05-24 14:26:20.186271981Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VogH8d_v03-pKxveKvcQRpR","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330380.342253_100.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:20.342253953Z","data":{"read_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":2},"status":{"status":"write"}}},{"id":"1_1590330380.583029_101.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:20.583029777Z","data":{"read_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330903.436321_102.1","section":"bucket.instance","name":"ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","timestamp":"2020-05-24 14:35:03.436321562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gAFo5R_jIeCyFq2LSsNo17_","ver":1},"status":{"status":"write"}}},{"id":"1_1590330903.717682_103.1","section":"bucket.instance","name":"ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","timestamp":"2020-05-24 14:35:03.717682381Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gAFo5R_jIeCyFq2LSsNo17_","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330903.810200_104.1","section":"bucket","name":"ajxnhd-20","timestamp":"2020-05-24 14:35:03.810200796Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zOYKFnexj8DFjS8jtUKG4DJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590330903.938053_105.1","section":"bucket","name":"ajxnhd-20","timestamp":"2020-05-24 14:35:03.938053252Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zOYKFnexj8DFjS8jtUKG4DJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331429.252015_120.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:49.252015956Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"status":{"status":"write"}}},{"id":"1_1590331429.406239_121.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:49.406239502Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331429.478659_122.1","section":"bucket","name":"ajxnhd-21","timestamp":"2020-05-24 14:43:49.478659851Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LhHMDRFw1MbeqVRBXiLVGqV","ver":1},"status":{"status":"write"}}},{"id":"1_1590331429.775227_123.1","section":"bucket","name":"ajxnhd-21","timestamp":"2020-05-24 14:43:49.775227278Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LhHMDRFw1MbeqVRBXiLVGqV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331430.455137_124.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:50.455137998Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"status":{"status":"write"}}},{"id":"1_1590331430.680723_125.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:50.680723110Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331430.776858_126.1","section":"bucket","name":"ajxnhd-22","timestamp":"2020-05-24 14:43:50.776858522Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zS0Qv2MgKEFW14qiK5JnkWJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590331430.897169_127.1","section":"bucket","name":"ajxnhd-22","timestamp":"2020-05-24 14:43:50.897169843Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zS0Qv2MgKEFW14qiK5JnkWJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331432.003953_128.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:52.003953530Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"status":{"status":"write"}}},{"id":"1_1590331432.142214_129.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:52.142214510Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331432.205496_130.1","section":"bucket","name":"ajxnhd-23","timestamp":"2020-05-24 14:43:52.205496460Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yrznneKnovuxIXNnXnO0uLI","ver":1},"status":{"status":"write"}}},{"id":"1_1590331432.337983_131.1","section":"bucket","name":"ajxnhd-23","timestamp":"2020-05-24 14:43:52.337983599Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yrznneKnovuxIXNnXnO0uLI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331433.714200_132.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:53.714200278Z","data":{"read_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":2},"status":{"status":"write"}}},{"id":"1_1590331433.948922_133.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:53.948922752Z","data":{"read_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331434.894781_134.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:54.894781681Z","data":{"read_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":2},"status":{"status":"write"}}},{"id":"1_1590331435.198978_135.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:55.198978175Z","data":{"read_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331435.928156_136.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:55.928156482Z","data":{"read_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":2},"status":{"status":"write"}}},{"id":"1_1590331436.244684_137.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:56.244684179Z","data":{"read_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331983.049122_144.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:03.049122099Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"status":{"status":"write"}}},{"id":"1_1590331983.197037_145.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:03.197037858Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331983.266790_146.1","section":"bucket","name":"ajxnhd-24","timestamp":"2020-05-24 14:53:03.266790936Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uIlRzMpJio6QSnre8AMGFLm","ver":1},"status":{"status":"write"}}},{"id":"1_1590331983.424889_147.1","section":"bucket","name":"ajxnhd-24","timestamp":"2020-05-24 14:53:03.424889595Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uIlRzMpJio6QSnre8AMGFLm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331983.865257_148.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:03.865257449Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"status":{"status":"write"}}},{"id":"1_1590331984.122968_149.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:04.122968508Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331984.251929_150.1","section":"bucket","name":"ajxnhd-25","timestamp":"2020-05-24 14:53:04.251929448Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uQJr9MjpeQurPAVQIXSW_JM","ver":1},"status":{"status":"write"}}},{"id":"1_1590331984.505736_151.1","section":"bucket","name":"ajxnhd-25","timestamp":"2020-05-24 14:53:04.505736130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uQJr9MjpeQurPAVQIXSW_JM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331986.438990_152.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:06.438990629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"status":{"status":"write"}}},{"id":"1_1590331986.585637_153.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:06.585637806Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331986.647556_154.1","section":"bucket","name":"ajxnhd-26","timestamp":"2020-05-24 14:53:06.647556290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Al_0YQq_jeAkOhBQMDkf2Z7","ver":1},"status":{"status":"write"}}},{"id":"1_1590331986.814186_155.1","section":"bucket","name":"ajxnhd-26","timestamp":"2020-05-24 14:53:06.814186094Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Al_0YQq_jeAkOhBQMDkf2Z7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331988.439385_156.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.439385111Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"status":{"status":"write"}}},{"id":"1_1590331988.794207_157.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.794207447Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331988.879023_158.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.879023435Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":3},"status":{"status":"write"}}},{"id":"1_1590331989.060960_159.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:09.060960790Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":3},"status":{"status":"complete"}}},{"id":"1_1590331989.527880_160.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:09.527880038Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"status":{"status":"write"}}},{"id":"1_1590331989.813285_161.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:09.813285672Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331990.123113_162.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:10.123113736Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":3},"status":{"status":"write"}}},{"id":"1_1590331990.429901_163.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:10.429901443Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":3},"status":{"status":"complete"}}},{"id":"1_1590331991.738750_164.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:11.738750891Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"status":{"status":"write"}}},{"id":"1_1590331991.926433_165.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:11.926433769Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331992.233007_166.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:12.233007240Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":3},"status":{"status":"write"}}},{"id":"1_1590331992.462010_167.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:12.462010498Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332542.631802_177.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:22.631802311Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"status":{"status":"write"}}},{"id":"1_1590332542.830206_178.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:22.830206865Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332542.936016_179.1","section":"bucket","name":"ajxnhd-27","timestamp":"2020-05-24 15:02:22.936016582Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6-ZQ1R4_Nnl5GdcjPwHQgoI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332543.103013_180.1","section":"bucket","name":"ajxnhd-27","timestamp":"2020-05-24 15:02:23.103013361Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6-ZQ1R4_Nnl5GdcjPwHQgoI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332543.775583_181.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:23.775583592Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"status":{"status":"write"}}},{"id":"1_1590332544.023476_182.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:24.023476172Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332544.120691_183.1","section":"bucket","name":"ajxnhd-28","timestamp":"2020-05-24 15:02:24.120691142Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_S_0X21LYEXlkSLS9-UznAcI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332544.302282_184.1","section":"bucket","name":"ajxnhd-28","timestamp":"2020-05-24 15:02:24.302282945Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_S_0X21LYEXlkSLS9-UznAcI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332545.404781_185.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:25.404781963Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332545.585733_186.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:25.585733302Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332545.654256_187.1","section":"bucket","name":"ajxnhd-29","timestamp":"2020-05-24 15:02:25.654256391Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vqZzPK8m08iLaKomZNlavH0","ver":1},"status":{"status":"write"}}},{"id":"1_1590332545.771078_188.1","section":"bucket","name":"ajxnhd-29","timestamp":"2020-05-24 15:02:25.771078802Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vqZzPK8m08iLaKomZNlavH0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332546.555221_189.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:26.555221665Z","data":{"read_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":2},"status":{"status":"write"}}},{"id":"1_1590332546.717101_190.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:26.717101024Z","data":{"read_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332547.859104_191.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:27.859104280Z","data":{"read_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":2},"status":{"status":"write"}}},{"id":"1_1590332548.198444_192.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:28.198444891Z","data":{"read_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332548.675933_193.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:28.675933514Z","data":{"read_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":2},"status":{"status":"write"}}},{"id":"1_1590332549.028549_194.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:29.028549384Z","data":{"read_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332549.967216_195.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:29.967216514Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"status":{"status":"write"}}},{"id":"1_1590332550.115459_196.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:30.115459573Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332550.244077_197.1","section":"bucket","name":"ajxnhd-30","timestamp":"2020-05-24 15:02:30.244077484Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P61wwDe_JM5YuqmV6LehKBK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332550.456855_198.1","section":"bucket","name":"ajxnhd-30","timestamp":"2020-05-24 15:02:30.456855328Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P61wwDe_JM5YuqmV6LehKBK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332551.822124_199.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:31.822124501Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"status":{"status":"write"}}},{"id":"1_1590332552.065161_200.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:32.065161497Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332552.144138_201.1","section":"bucket","name":"ajxnhd-31","timestamp":"2020-05-24 15:02:32.144138917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_guBR1c3-SAHv8LKMh2QLb7A","ver":1},"status":{"status":"write"}}},{"id":"1_1590332552.260354_202.1","section":"bucket","name":"ajxnhd-31","timestamp":"2020-05-24 15:02:32.260354003Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_guBR1c3-SAHv8LKMh2QLb7A","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332553.270432_203.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:33.270432895Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"status":{"status":"write"}}},{"id":"1_1590332553.415925_204.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:33.415925128Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332553.486668_205.1","section":"bucket","name":"ajxnhd-32","timestamp":"2020-05-24 15:02:33.486668591Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_s1Byxm5NDVFw_jUWDF_FjgS","ver":1},"status":{"status":"write"}}},{"id":"1_1590332553.660263_206.1","section":"bucket","name":"ajxnhd-32","timestamp":"2020-05-24 15:02:33.660263701Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_s1Byxm5NDVFw_jUWDF_FjgS","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332554.995884_207.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:34.995884736Z","data":{"read_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":2},"status":{"status":"write"}}},{"id":"1_1590332555.358899_208.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:35.358899184Z","data":{"read_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332556.426757_209.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:36.426757678Z","data":{"read_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":2},"status":{"status":"write"}}},{"id":"1_1590332556.687130_210.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:36.687130128Z","data":{"read_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332557.695414_211.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:37.695414905Z","data":{"read_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":2},"status":{"status":"write"}}},{"id":"1_1590332557.981422_212.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:37.981422719Z","data":{"read_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332559.305198_213.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:39.305198079Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"status":{"status":"write"}}},{"id":"1_1590332559.482651_214.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:39.482651412Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332559.540073_215.1","section":"bucket","name":"ajxnhd-33","timestamp":"2020-05-24 15:02:39.540073474Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__TXIL7F0fo6f_ML5KkyR5Cf","ver":1},"status":{"status":"write"}}},{"id":"1_1590332559.677626_216.1","section":"bucket","name":"ajxnhd-33","timestamp":"2020-05-24 15:02:39.677626433Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__TXIL7F0fo6f_ML5KkyR5Cf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332560.881764_217.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:40.881764475Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"status":{"status":"write"}}},{"id":"1_1590332561.064185_218.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:41.064185535Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332561.144892_219.1","section":"bucket","name":"ajxnhd-34","timestamp":"2020-05-24 15:02:41.144892141Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TnCet-EC2ujq4RjCMmtoi5b","ver":1},"status":{"status":"write"}}},{"id":"1_1590332561.397068_220.1","section":"bucket","name":"ajxnhd-34","timestamp":"2020-05-24 15:02:41.397068589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TnCet-EC2ujq4RjCMmtoi5b","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332562.761328_221.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:42.761328654Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"status":{"status":"write"}}},{"id":"1_1590332562.887604_222.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:42.887604521Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332562.947140_223.1","section":"bucket","name":"ajxnhd-35","timestamp":"2020-05-24 15:02:42.947140222Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4J9adjb6Qa0J6bXq6rh1nYz","ver":1},"status":{"status":"write"}}},{"id":"1_1590332563.062797_224.1","section":"bucket","name":"ajxnhd-35","timestamp":"2020-05-24 15:02:43.062797471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4J9adjb6Qa0J6bXq6rh1nYz","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332564.442927_225.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:44.442927388Z","data":{"read_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":2},"status":{"status":"write"}}},{"id":"1_1590332564.792134_226.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:44.792134507Z","data":{"read_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332566.229361_227.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:46.229361522Z","data":{"read_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":2},"status":{"status":"write"}}},{"id":"1_1590332566.526084_228.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:46.526084519Z","data":{"read_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332568.689676_229.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:48.689676599Z","data":{"read_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":2},"status":{"status":"write"}}},{"id":"1_1590332568.878454_230.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:48.878454727Z","data":{"read_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332570.049297_231.1","section":"bucket.instance","name":"ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","timestamp":"2020-05-24 15:02:50.049297529Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_b9k2iMA1NtKXNRMZLjKnuMK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332570.244663_232.1","section":"bucket.instance","name":"ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","timestamp":"2020-05-24 15:02:50.244663844Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_b9k2iMA1NtKXNRMZLjKnuMK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332570.372091_233.1","section":"bucket","name":"ajxnhd-36","timestamp":"2020-05-24 15:02:50.372091228Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7Jm22K8v7V5ENeX7fvvO6m3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332570.524216_234.1","section":"bucket","name":"ajxnhd-36","timestamp":"2020-05-24 15:02:50.524216691Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7Jm22K8v7V5ENeX7fvvO6m3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332570.961122_235.1","section":"bucket.instance","name":"ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","timestamp":"2020-05-24 15:02:50.961122188Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_24WmxNrnrGYWY6wlu6B-ieA","ver":1},"status":{"status":"write"}}},{"id":"1_1590332571.260025_236.1","section":"bucket.instance","name":"ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","timestamp":"2020-05-24 15:02:51.260025363Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_24WmxNrnrGYWY6wlu6B-ieA","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332571.358013_237.1","section":"bucket","name":"ajxnhd-37","timestamp":"2020-05-24 15:02:51.358013136Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sEb0Ve1wRSuEJNe56JIG4EI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332571.530066_238.1","section":"bucket","name":"ajxnhd-37","timestamp":"2020-05-24 15:02:51.530066130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sEb0Ve1wRSuEJNe56JIG4EI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332573.119556_239.1","section":"bucket.instance","name":"ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","timestamp":"2020-05-24 15:02:53.119556405Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7hfIFt8sICEW9pyMqr-MYV","ver":1},"status":{"status":"write"}}},{"id":"1_1590332573.260989_240.1","section":"bucket.instance","name":"ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","timestamp":"2020-05-24 15:02:53.260989054Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7hfIFt8sICEW9pyMqr-MYV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332573.335527_241.1","section":"bucket","name":"ajxnhd-38","timestamp":"2020-05-24 15:02:53.335527083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vNQMrHjRLeJ3QZbyKgH0O_6","ver":1},"status":{"status":"write"}}},{"id":"1_1590332573.500710_242.1","section":"bucket","name":"ajxnhd-38","timestamp":"2020-05-24 15:02:53.500710562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vNQMrHjRLeJ3QZbyKgH0O_6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.965748_243.1","section":"bucket.instance","name":"ajxnhd-39:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48","timestamp":"2020-05-24 15:03:11.965748944Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y58mPDPi7qXqG_CmkdmR6os","ver":1},"status":{"status":"write"}}},{"id":"1_1590332592.803259_244.1","section":"bucket.instance","name":"ajxnhd-39:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48","timestamp":"2020-05-24 15:03:12.803259188Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y58mPDPi7qXqG_CmkdmR6os","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.312415_245.1","section":"bucket","name":"ajxnhd-39","timestamp":"2020-05-24 15:03:13.312415629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UNwrQUdBwy_B1nsJ18BGrXu","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.915640_246.1","section":"bucket","name":"ajxnhd-39","timestamp":"2020-05-24 15:03:13.915640047Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UNwrQUdBwy_B1nsJ18BGrXu","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332595.681045_247.1","section":"bucket.instance","name":"ajxnhd-40:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49","timestamp":"2020-05-24 15:03:15.681045977Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cGbybOYRxqaUODITnQ64wX8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332595.965278_248.1","section":"bucket.instance","name":"ajxnhd-40:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49","timestamp":"2020-05-24 15:03:15.965278368Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cGbybOYRxqaUODITnQ64wX8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332596.121270_249.1","section":"bucket","name":"ajxnhd-40","timestamp":"2020-05-24 15:03:16.121270922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WSY76UVteF1dCj_5H4jHX5D","ver":1},"status":{"status":"write"}}},{"id":"1_1590332596.288693_250.1","section":"bucket","name":"ajxnhd-40","timestamp":"2020-05-24 15:03:16.288693632Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WSY76UVteF1dCj_5H4jHX5D","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332597.688431_251.1","section":"bucket.instance","name":"ajxnhd-41:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50","timestamp":"2020-05-24 15:03:17.688431472Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AK-EzZL3ZkVBaaCNyKRG8cA","ver":1},"status":{"status":"write"}}},{"id":"1_1590332597.968361_252.1","section":"bucket.instance","name":"ajxnhd-41:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50","timestamp":"2020-05-24 15:03:17.968361011Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AK-EzZL3ZkVBaaCNyKRG8cA","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332598.088015_253.1","section":"bucket","name":"ajxnhd-41","timestamp":"2020-05-24 15:03:18.088015374Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gs-0jKiD8zj7WuELPIX45Fk","ver":1},"status":{"status":"write"}}},{"id":"1_1590332598.292976_254.1","section":"bucket","name":"ajxnhd-41","timestamp":"2020-05-24 15:03:18.292976652Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gs-0jKiD8zj7WuELPIX45Fk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328661.857046_44.1","section":"bucket.instance","name":"ajxnhd-1:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1","timestamp":"2020-05-24 13:57:41.857046576Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EI_f_0Ki9favYOjyGg-58P0","ver":1},"status":{"status":"write"}}},{"id":"1_1590328663.801409_45.1","section":"bucket.instance","name":"ajxnhd-1:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1","timestamp":"2020-05-24 13:57:43.801409385Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EI_f_0Ki9favYOjyGg-58P0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328665.095008_46.1","section":"bucket","name":"ajxnhd-1","timestamp":"2020-05-24 13:57:45.095008350Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZoWBta9CxnaXqf3Hnfk32TH","ver":1},"status":{"status":"write"}}},{"id":"1_1590328666.568066_47.1","section":"bucket","name":"ajxnhd-1","timestamp":"2020-05-24 13:57:46.568066655Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZoWBta9CxnaXqf3Hnfk32TH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328669.388156_48.1","section":"bucket.instance","name":"ajxnhd-2:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2","timestamp":"2020-05-24 13:57:49.388156020Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GLHHyqK2R6W2PTMOkPNrrlk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328671.962074_49.1","section":"bucket.instance","name":"ajxnhd-2:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2","timestamp":"2020-05-24 13:57:51.962074124Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GLHHyqK2R6W2PTMOkPNrrlk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328674.256018_50.1","section":"bucket","name":"ajxnhd-2","timestamp":"2020-05-24 13:57:54.256018718Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGN9Rv5tiGtmUXakupLtXPZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328676.357935_51.1","section":"bucket","name":"ajxnhd-2","timestamp":"2020-05-24 13:57:56.357935846Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGN9Rv5tiGtmUXakupLtXPZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328681.147077_52.1","section":"bucket.instance","name":"ajxnhd-3:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3","timestamp":"2020-05-24 13:58:01.147077733Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9TpTYOx5Q03ed_c9goFYJZM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328681.299764_53.1","section":"bucket.instance","name":"ajxnhd-3:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3","timestamp":"2020-05-24 13:58:01.299764826Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9TpTYOx5Q03ed_c9goFYJZM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328681.371143_54.1","section":"bucket","name":"ajxnhd-3","timestamp":"2020-05-24 13:58:01.371143798Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vPul70r_8Ep8FRLwG3vWyNZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328681.510269_55.1","section":"bucket","name":"ajxnhd-3","timestamp":"2020-05-24 13:58:01.510269305Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vPul70r_8Ep8FRLwG3vWyNZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328695.908700_56.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4","timestamp":"2020-05-24 13:58:15.908700393Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vLXsYr9SmfTZ3Fb3d_wnAd3","ver":1},"status":{"status":"write"}}},{"id":"1_1590328696.114303_57.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4","timestamp":"2020-05-24 13:58:16.114303048Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vLXsYr9SmfTZ3Fb3d_wnAd3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328696.194626_58.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:16.194626894Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Jg0IeWtO91dabgJ-_9OekxY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328696.347412_59.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:16.347412194Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Jg0IeWtO91dabgJ-_9OekxY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328697.502176_60.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5","timestamp":"2020-05-24 13:58:17.502176684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LqXlxfl8QRw7-uDVpsgnDcw","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.668291_61.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5","timestamp":"2020-05-24 13:58:17.668291982Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LqXlxfl8QRw7-uDVpsgnDcw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328697.761208_62.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:17.761208203Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_obyOfpY3MrTzY_e23jlE70I","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.902648_63.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:17.902648222Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_obyOfpY3MrTzY_e23jlE70I","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328699.527169_64.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6","timestamp":"2020-05-24 13:58:19.527169223Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_o4Ky02fbN6YWn_f_ggrN7C7","ver":1},"status":{"status":"write"}}},{"id":"1_1590328699.677825_65.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6","timestamp":"2020-05-24 13:58:19.677825821Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_o4Ky02fbN6YWn_f_ggrN7C7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328699.833869_66.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:19.833869190Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_enfzxiViqv_w3BWd10aOZ0d","ver":1},"status":{"status":"write"}}},{"id":"1_1590328700.029897_67.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:20.029897018Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_enfzxiViqv_w3BWd10aOZ0d","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328714.515118_70.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.515118726Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"status":{"status":"write"}}},{"id":"1_1590328714.719389_71.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.719389695Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328714.782348_72.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:34.782348779Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4HVX6Tlmf_oEsDt4DBAWClI","ver":1},"status":{"status":"write"}}},{"id":"1_1590328714.897108_73.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:34.897108137Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4HVX6Tlmf_oEsDt4DBAWClI","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328714.993989_74.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.993989083Z","data":{"read_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328715.172258_75.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:35.172258827Z","data":{"read_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328715.564378_76.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:35.564378963Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"status":{"status":"write"}}},{"id":"1_1590328715.749934_77.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:35.749934229Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328715.857497_78.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:35.857497048Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NJuC_NimR8vZNocPMx3wfsi","ver":1},"status":{"status":"write"}}},{"id":"1_1590328716.050040_79.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:36.050040815Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NJuC_NimR8vZNocPMx3wfsi","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328716.187601_80.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:36.187601735Z","data":{"read_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328716.382658_81.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:36.382658090Z","data":{"read_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328716.732842_82.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:36.732842955Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328717.017969_83.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.017969634Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328717.178040_84.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:37.178040274Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sCm2MGtfgkAIqwwkQk3Yw5q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328717.373287_85.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:37.373287652Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sCm2MGtfgkAIqwwkQk3Yw5q","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328717.510560_86.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.510560445Z","data":{"read_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328717.684607_87.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.684607007Z","data":{"read_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328718.317405_89.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.317405148Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"status":{"status":"write"}}},{"id":"1_1590328718.504832_90.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.504832890Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328718.571111_91.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:38.571111899Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e8rj40fpi3-JZbsFLxS45Xk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328718.700562_92.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:38.700562070Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e8rj40fpi3-JZbsFLxS45Xk","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328718.860296_93.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.860296964Z","data":{"read_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328719.136857_94.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:39.136857916Z","data":{"read_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328720.315474_95.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.315474784Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"status":{"status":"write"}}},{"id":"1_1590328720.499875_96.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.499875922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328720.583970_97.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:40.583970910Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xi3oOUty4bv3bHoncH0dIz3","ver":1},"status":{"status":"write"}}},{"id":"1_1590328720.753543_98.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:40.753543145Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xi3oOUty4bv3bHoncH0dIz3","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328720.994211_99.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.994211183Z","data":{"read_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328721.267286_100.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:41.267286430Z","data":{"read_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328722.062273_101.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.062273899Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328722.334638_102.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.334638645Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328722.416735_103.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:42.416735183Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_dVkfYwfTCvS5gmH63FscSTy","ver":1},"status":{"status":"write"}}},{"id":"1_1590328722.542035_104.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:42.542035756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_dVkfYwfTCvS5gmH63FscSTy","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328722.640315_105.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.640315136Z","data":{"read_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328722.943793_106.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.943793752Z","data":{"read_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328724.614660_108.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:44.614660660Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328724.809740_109.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:44.809740816Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328724.882230_110.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:44.882230120Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UlCm7CE1xo5AsxtoMJbTMz4","ver":1},"status":{"status":"write"}}},{"id":"1_1590328725.087079_111.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:45.087079418Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UlCm7CE1xo5AsxtoMJbTMz4","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328725.239107_112.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:45.239107841Z","data":{"read_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328725.441830_113.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:45.441830031Z","data":{"read_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328726.292849_114.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.292849280Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"status":{"status":"write"}}},{"id":"1_1590328726.443715_115.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.443715821Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328726.524327_116.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:46.524327926Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VBLmoFhcIYIVqpzreE0XnKi","ver":1},"status":{"status":"write"}}},{"id":"1_1590328726.674271_117.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:46.674271737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VBLmoFhcIYIVqpzreE0XnKi","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328726.787812_118.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.787812842Z","data":{"read_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328727.010860_119.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:47.010860196Z","data":{"read_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328727.922311_120.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:47.922311552Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328728.057327_121.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.057327847Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328728.160326_122.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:48.160326113Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u5I6Hkn6tF0zFwYrBt4lf2D","ver":1},"status":{"status":"write"}}},{"id":"1_1590328728.345721_123.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:48.345721203Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u5I6Hkn6tF0zFwYrBt4lf2D","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328728.436742_124.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.436742651Z","data":{"read_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328728.582516_125.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.582516268Z","data":{"read_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328744.824204_129.1","section":"bucket.instance","name":"ajxnhd-7:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16","timestamp":"2020-05-24 13:59:04.824204578Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_azYpBimd6nndafzvDBhV2e6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328745.009557_130.1","section":"bucket.instance","name":"ajxnhd-7:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16","timestamp":"2020-05-24 13:59:05.009557060Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_azYpBimd6nndafzvDBhV2e6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328745.089321_131.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:05.089321631Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328745.283838_132.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:05.283838511Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328746.557992_133.1","section":"bucket.instance","name":"ajxnhd-8:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17","timestamp":"2020-05-24 13:59:06.557992864Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iJutw-w353HxM0wQe0ahGW0","ver":1},"status":{"status":"write"}}},{"id":"1_1590328746.745740_134.1","section":"bucket.instance","name":"ajxnhd-8:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17","timestamp":"2020-05-24 13:59:06.745740588Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iJutw-w353HxM0wQe0ahGW0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328746.807024_135.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:06.807024179Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"status":{"status":"write"}}},{"id":"1_1590328747.035908_136.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:07.035908477Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328749.986118_137.1","section":"bucket.instance","name":"ajxnhd-9:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18","timestamp":"2020-05-24 13:59:09.986118602Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_k86zaCt0pVAFBXphA-HHAKO","ver":1},"status":{"status":"write"}}},{"id":"1_1590328750.203959_138.1","section":"bucket.instance","name":"ajxnhd-9:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18","timestamp":"2020-05-24 13:59:10.203959616Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_k86zaCt0pVAFBXphA-HHAKO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328750.291133_139.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:10.291133261Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"status":{"status":"write"}}},{"id":"1_1590328750.444965_140.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:10.444965711Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328764.224998_141.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:24.224998965Z","data":{"read_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328764.442285_142.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:24.442285080Z","data":{"read_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328765.812508_143.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:25.812508979Z","data":{"read_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328765.976327_144.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:25.976327786Z","data":{"read_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328767.020248_145.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:27.020248606Z","data":{"read_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328767.216616_146.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:27.216616731Z","data":{"read_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":2},"status":{"status":"complete"}}}] 2020-05-24T15:06:46.768 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:46.768 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:46.769 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 114a3cfc-32a6-4ae9-bab5-baac4f59a79e --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:49.937 INFO:teuthology.orchestra.run.smithi160.stdout:[{"id":"1_1590328567.776739_38.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:07.776739104Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ui15Zsog5OeOBfQLFnhWmAY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328568.276896_39.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:08.276896545Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ui15Zsog5OeOBfQLFnhWmAY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328780.710920_36.1","section":"bucket.instance","name":"ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","timestamp":"2020-05-24 13:59:40.710920367Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_1x1ZZjoXeMNzJoUjkg5bWmM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328780.907463_37.1","section":"bucket.instance","name":"ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","timestamp":"2020-05-24 13:59:40.907463503Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_1x1ZZjoXeMNzJoUjkg5bWmM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328780.992516_38.1","section":"bucket","name":"ajxnhd-10","timestamp":"2020-05-24 13:59:40.992516918Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_F0eV8p4gy4VVoOyBszXK9Uy","ver":1},"status":{"status":"write"}}},{"id":"1_1590328781.173823_39.1","section":"bucket","name":"ajxnhd-10","timestamp":"2020-05-24 13:59:41.173823390Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_F0eV8p4gy4VVoOyBszXK9Uy","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328781.786763_40.1","section":"bucket.instance","name":"ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","timestamp":"2020-05-24 13:59:41.786763813Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yAfrDCGApzmfUfpLV32eRYD","ver":1},"status":{"status":"write"}}},{"id":"1_1590328782.049055_41.1","section":"bucket.instance","name":"ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","timestamp":"2020-05-24 13:59:42.049055331Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yAfrDCGApzmfUfpLV32eRYD","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328782.155023_42.1","section":"bucket","name":"ajxnhd-11","timestamp":"2020-05-24 13:59:42.155023264Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6JZ7TwZSk3ER0WTRkMdpYxK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328782.443391_43.1","section":"bucket","name":"ajxnhd-11","timestamp":"2020-05-24 13:59:42.443391351Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6JZ7TwZSk3ER0WTRkMdpYxK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.411331_44.1","section":"bucket.instance","name":"ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","timestamp":"2020-05-24 13:59:43.411331039Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5HqOnT066cNskmDRsRqZ-dE","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.597981_45.1","section":"bucket.instance","name":"ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","timestamp":"2020-05-24 13:59:43.597981263Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5HqOnT066cNskmDRsRqZ-dE","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.683466_46.1","section":"bucket","name":"ajxnhd-12","timestamp":"2020-05-24 13:59:43.683466272Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_z0ez1Ljq8OH081qJv8Q7CL6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.837755_47.1","section":"bucket","name":"ajxnhd-12","timestamp":"2020-05-24 13:59:43.837755756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_z0ez1Ljq8OH081qJv8Q7CL6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329320.125728_54.1","section":"bucket.instance","name":"ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","timestamp":"2020-05-24 14:08:40.125728151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NEqYrxBauZKMtGHHXFkIZCq","ver":1},"status":{"status":"write"}}},{"id":"1_1590329320.290517_55.1","section":"bucket.instance","name":"ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","timestamp":"2020-05-24 14:08:40.290517017Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NEqYrxBauZKMtGHHXFkIZCq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329320.400997_56.1","section":"bucket","name":"ajxnhd-13","timestamp":"2020-05-24 14:08:40.400997847Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eweXCos3PdlUyYekfNFjSHH","ver":1},"status":{"status":"write"}}},{"id":"1_1590329320.572773_57.1","section":"bucket","name":"ajxnhd-13","timestamp":"2020-05-24 14:08:40.572773700Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eweXCos3PdlUyYekfNFjSHH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329322.157703_58.1","section":"bucket.instance","name":"ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","timestamp":"2020-05-24 14:08:42.157703103Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BmyQDJAOboCFiJ45lLtofKE","ver":1},"status":{"status":"write"}}},{"id":"1_1590329322.303444_59.1","section":"bucket.instance","name":"ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","timestamp":"2020-05-24 14:08:42.303444730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BmyQDJAOboCFiJ45lLtofKE","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329322.485950_60.1","section":"bucket","name":"ajxnhd-14","timestamp":"2020-05-24 14:08:42.485950917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_quzz_j7V4et7RpGVfsZ8DI3","ver":1},"status":{"status":"write"}}},{"id":"1_1590329322.753097_61.1","section":"bucket","name":"ajxnhd-14","timestamp":"2020-05-24 14:08:42.753097194Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_quzz_j7V4et7RpGVfsZ8DI3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329323.912546_62.1","section":"bucket.instance","name":"ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","timestamp":"2020-05-24 14:08:43.912546421Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ljytbx-c-ioF_jOyI19KhIn","ver":1},"status":{"status":"write"}}},{"id":"1_1590329324.055027_63.1","section":"bucket.instance","name":"ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","timestamp":"2020-05-24 14:08:44.055027596Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ljytbx-c-ioF_jOyI19KhIn","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329324.140936_64.1","section":"bucket","name":"ajxnhd-15","timestamp":"2020-05-24 14:08:44.140936259Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__NTNWGbez_pXSCfSDkXBqXJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590329324.279876_65.1","section":"bucket","name":"ajxnhd-15","timestamp":"2020-05-24 14:08:44.279876745Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__NTNWGbez_pXSCfSDkXBqXJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329851.002187_72.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:31.002187986Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"status":{"status":"write"}}},{"id":"1_1590329851.192404_73.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:31.192404499Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329851.255970_74.1","section":"bucket","name":"ajxnhd-16","timestamp":"2020-05-24 14:17:31.255970913Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_n5pa9q08aZnglX0ZBX2MI72","ver":1},"status":{"status":"write"}}},{"id":"1_1590329851.450578_75.1","section":"bucket","name":"ajxnhd-16","timestamp":"2020-05-24 14:17:31.450578151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_n5pa9q08aZnglX0ZBX2MI72","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329852.156682_76.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:32.156682053Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"status":{"status":"write"}}},{"id":"1_1590329852.314606_77.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:32.314606785Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329852.385946_78.1","section":"bucket","name":"ajxnhd-17","timestamp":"2020-05-24 14:17:32.385946764Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2yVMkwTR0dwxoaxsJDfspmT","ver":1},"status":{"status":"write"}}},{"id":"1_1590329852.537860_79.1","section":"bucket","name":"ajxnhd-17","timestamp":"2020-05-24 14:17:32.537860267Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2yVMkwTR0dwxoaxsJDfspmT","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329853.921024_80.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:33.921024364Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"status":{"status":"write"}}},{"id":"1_1590329854.087205_81.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:34.087205756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329854.204871_82.1","section":"bucket","name":"ajxnhd-18","timestamp":"2020-05-24 14:17:34.204871979Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBs2gR836NQmjWKNfHbjG8K","ver":1},"status":{"status":"write"}}},{"id":"1_1590329854.445256_83.1","section":"bucket","name":"ajxnhd-18","timestamp":"2020-05-24 14:17:34.445256737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBs2gR836NQmjWKNfHbjG8K","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329855.079839_84.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:35.079839696Z","data":{"read_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":2},"status":{"status":"write"}}},{"id":"1_1590329855.330800_85.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:35.330800723Z","data":{"read_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329856.118614_86.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:36.118614343Z","data":{"read_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":2},"status":{"status":"write"}}},{"id":"1_1590329856.475914_87.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:36.475914391Z","data":{"read_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329857.623824_88.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:37.623824536Z","data":{"read_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":2},"status":{"status":"write"}}},{"id":"1_1590329857.827328_89.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:37.827328629Z","data":{"read_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330379.734826_96.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:19.734826079Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"status":{"status":"write"}}},{"id":"1_1590330379.889098_97.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:19.889098852Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330380.055101_98.1","section":"bucket","name":"ajxnhd-19","timestamp":"2020-05-24 14:26:20.055101601Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VogH8d_v03-pKxveKvcQRpR","ver":1},"status":{"status":"write"}}},{"id":"1_1590330380.186271_99.1","section":"bucket","name":"ajxnhd-19","timestamp":"2020-05-24 14:26:20.186271981Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VogH8d_v03-pKxveKvcQRpR","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330380.342253_100.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:20.342253953Z","data":{"read_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":2},"status":{"status":"write"}}},{"id":"1_1590330380.583029_101.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:20.583029777Z","data":{"read_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330903.436321_102.1","section":"bucket.instance","name":"ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","timestamp":"2020-05-24 14:35:03.436321562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gAFo5R_jIeCyFq2LSsNo17_","ver":1},"status":{"status":"write"}}},{"id":"1_1590330903.717682_103.1","section":"bucket.instance","name":"ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","timestamp":"2020-05-24 14:35:03.717682381Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gAFo5R_jIeCyFq2LSsNo17_","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330903.810200_104.1","section":"bucket","name":"ajxnhd-20","timestamp":"2020-05-24 14:35:03.810200796Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zOYKFnexj8DFjS8jtUKG4DJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590330903.938053_105.1","section":"bucket","name":"ajxnhd-20","timestamp":"2020-05-24 14:35:03.938053252Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zOYKFnexj8DFjS8jtUKG4DJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331429.252015_120.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:49.252015956Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"status":{"status":"write"}}},{"id":"1_1590331429.406239_121.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:49.406239502Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331429.478659_122.1","section":"bucket","name":"ajxnhd-21","timestamp":"2020-05-24 14:43:49.478659851Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LhHMDRFw1MbeqVRBXiLVGqV","ver":1},"status":{"status":"write"}}},{"id":"1_1590331429.775227_123.1","section":"bucket","name":"ajxnhd-21","timestamp":"2020-05-24 14:43:49.775227278Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LhHMDRFw1MbeqVRBXiLVGqV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331430.455137_124.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:50.455137998Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"status":{"status":"write"}}},{"id":"1_1590331430.680723_125.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:50.680723110Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331430.776858_126.1","section":"bucket","name":"ajxnhd-22","timestamp":"2020-05-24 14:43:50.776858522Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zS0Qv2MgKEFW14qiK5JnkWJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590331430.897169_127.1","section":"bucket","name":"ajxnhd-22","timestamp":"2020-05-24 14:43:50.897169843Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zS0Qv2MgKEFW14qiK5JnkWJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331432.003953_128.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:52.003953530Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"status":{"status":"write"}}},{"id":"1_1590331432.142214_129.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:52.142214510Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331432.205496_130.1","section":"bucket","name":"ajxnhd-23","timestamp":"2020-05-24 14:43:52.205496460Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yrznneKnovuxIXNnXnO0uLI","ver":1},"status":{"status":"write"}}},{"id":"1_1590331432.337983_131.1","section":"bucket","name":"ajxnhd-23","timestamp":"2020-05-24 14:43:52.337983599Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yrznneKnovuxIXNnXnO0uLI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331433.714200_132.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:53.714200278Z","data":{"read_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":2},"status":{"status":"write"}}},{"id":"1_1590331433.948922_133.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:53.948922752Z","data":{"read_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331434.894781_134.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:54.894781681Z","data":{"read_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":2},"status":{"status":"write"}}},{"id":"1_1590331435.198978_135.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:55.198978175Z","data":{"read_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331435.928156_136.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:55.928156482Z","data":{"read_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":2},"status":{"status":"write"}}},{"id":"1_1590331436.244684_137.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:56.244684179Z","data":{"read_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331983.049122_144.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:03.049122099Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"status":{"status":"write"}}},{"id":"1_1590331983.197037_145.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:03.197037858Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331983.266790_146.1","section":"bucket","name":"ajxnhd-24","timestamp":"2020-05-24 14:53:03.266790936Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uIlRzMpJio6QSnre8AMGFLm","ver":1},"status":{"status":"write"}}},{"id":"1_1590331983.424889_147.1","section":"bucket","name":"ajxnhd-24","timestamp":"2020-05-24 14:53:03.424889595Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uIlRzMpJio6QSnre8AMGFLm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331983.865257_148.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:03.865257449Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"status":{"status":"write"}}},{"id":"1_1590331984.122968_149.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:04.122968508Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331984.251929_150.1","section":"bucket","name":"ajxnhd-25","timestamp":"2020-05-24 14:53:04.251929448Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uQJr9MjpeQurPAVQIXSW_JM","ver":1},"status":{"status":"write"}}},{"id":"1_1590331984.505736_151.1","section":"bucket","name":"ajxnhd-25","timestamp":"2020-05-24 14:53:04.505736130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uQJr9MjpeQurPAVQIXSW_JM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331986.438990_152.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:06.438990629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"status":{"status":"write"}}},{"id":"1_1590331986.585637_153.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:06.585637806Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331986.647556_154.1","section":"bucket","name":"ajxnhd-26","timestamp":"2020-05-24 14:53:06.647556290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Al_0YQq_jeAkOhBQMDkf2Z7","ver":1},"status":{"status":"write"}}},{"id":"1_1590331986.814186_155.1","section":"bucket","name":"ajxnhd-26","timestamp":"2020-05-24 14:53:06.814186094Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Al_0YQq_jeAkOhBQMDkf2Z7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331988.439385_156.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.439385111Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"status":{"status":"write"}}},{"id":"1_1590331988.794207_157.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.794207447Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331988.879023_158.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.879023435Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":3},"status":{"status":"write"}}},{"id":"1_1590331989.060960_159.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:09.060960790Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":3},"status":{"status":"complete"}}},{"id":"1_1590331989.527880_160.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:09.527880038Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"status":{"status":"write"}}},{"id":"1_1590331989.813285_161.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:09.813285672Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331990.123113_162.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:10.123113736Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":3},"status":{"status":"write"}}},{"id":"1_1590331990.429901_163.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:10.429901443Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":3},"status":{"status":"complete"}}},{"id":"1_1590331991.738750_164.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:11.738750891Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"status":{"status":"write"}}},{"id":"1_1590331991.926433_165.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:11.926433769Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331992.233007_166.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:12.233007240Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":3},"status":{"status":"write"}}},{"id":"1_1590331992.462010_167.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:12.462010498Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332542.631802_177.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:22.631802311Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"status":{"status":"write"}}},{"id":"1_1590332542.830206_178.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:22.830206865Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332542.936016_179.1","section":"bucket","name":"ajxnhd-27","timestamp":"2020-05-24 15:02:22.936016582Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6-ZQ1R4_Nnl5GdcjPwHQgoI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332543.103013_180.1","section":"bucket","name":"ajxnhd-27","timestamp":"2020-05-24 15:02:23.103013361Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6-ZQ1R4_Nnl5GdcjPwHQgoI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332543.775583_181.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:23.775583592Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"status":{"status":"write"}}},{"id":"1_1590332544.023476_182.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:24.023476172Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332544.120691_183.1","section":"bucket","name":"ajxnhd-28","timestamp":"2020-05-24 15:02:24.120691142Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_S_0X21LYEXlkSLS9-UznAcI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332544.302282_184.1","section":"bucket","name":"ajxnhd-28","timestamp":"2020-05-24 15:02:24.302282945Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_S_0X21LYEXlkSLS9-UznAcI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332545.404781_185.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:25.404781963Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332545.585733_186.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:25.585733302Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332545.654256_187.1","section":"bucket","name":"ajxnhd-29","timestamp":"2020-05-24 15:02:25.654256391Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vqZzPK8m08iLaKomZNlavH0","ver":1},"status":{"status":"write"}}},{"id":"1_1590332545.771078_188.1","section":"bucket","name":"ajxnhd-29","timestamp":"2020-05-24 15:02:25.771078802Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vqZzPK8m08iLaKomZNlavH0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332546.555221_189.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:26.555221665Z","data":{"read_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":2},"status":{"status":"write"}}},{"id":"1_1590332546.717101_190.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:26.717101024Z","data":{"read_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332547.859104_191.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:27.859104280Z","data":{"read_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":2},"status":{"status":"write"}}},{"id":"1_1590332548.198444_192.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:28.198444891Z","data":{"read_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332548.675933_193.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:28.675933514Z","data":{"read_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":2},"status":{"status":"write"}}},{"id":"1_1590332549.028549_194.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:29.028549384Z","data":{"read_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332549.967216_195.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:29.967216514Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"status":{"status":"write"}}},{"id":"1_1590332550.115459_196.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:30.115459573Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332550.244077_197.1","section":"bucket","name":"ajxnhd-30","timestamp":"2020-05-24 15:02:30.244077484Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P61wwDe_JM5YuqmV6LehKBK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332550.456855_198.1","section":"bucket","name":"ajxnhd-30","timestamp":"2020-05-24 15:02:30.456855328Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P61wwDe_JM5YuqmV6LehKBK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332551.822124_199.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:31.822124501Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"status":{"status":"write"}}},{"id":"1_1590332552.065161_200.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:32.065161497Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332552.144138_201.1","section":"bucket","name":"ajxnhd-31","timestamp":"2020-05-24 15:02:32.144138917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_guBR1c3-SAHv8LKMh2QLb7A","ver":1},"status":{"status":"write"}}},{"id":"1_1590332552.260354_202.1","section":"bucket","name":"ajxnhd-31","timestamp":"2020-05-24 15:02:32.260354003Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_guBR1c3-SAHv8LKMh2QLb7A","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332553.270432_203.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:33.270432895Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"status":{"status":"write"}}},{"id":"1_1590332553.415925_204.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:33.415925128Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332553.486668_205.1","section":"bucket","name":"ajxnhd-32","timestamp":"2020-05-24 15:02:33.486668591Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_s1Byxm5NDVFw_jUWDF_FjgS","ver":1},"status":{"status":"write"}}},{"id":"1_1590332553.660263_206.1","section":"bucket","name":"ajxnhd-32","timestamp":"2020-05-24 15:02:33.660263701Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_s1Byxm5NDVFw_jUWDF_FjgS","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332554.995884_207.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:34.995884736Z","data":{"read_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":2},"status":{"status":"write"}}},{"id":"1_1590332555.358899_208.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:35.358899184Z","data":{"read_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332556.426757_209.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:36.426757678Z","data":{"read_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":2},"status":{"status":"write"}}},{"id":"1_1590332556.687130_210.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:36.687130128Z","data":{"read_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332557.695414_211.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:37.695414905Z","data":{"read_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":2},"status":{"status":"write"}}},{"id":"1_1590332557.981422_212.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:37.981422719Z","data":{"read_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332559.305198_213.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:39.305198079Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"status":{"status":"write"}}},{"id":"1_1590332559.482651_214.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:39.482651412Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332559.540073_215.1","section":"bucket","name":"ajxnhd-33","timestamp":"2020-05-24 15:02:39.540073474Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__TXIL7F0fo6f_ML5KkyR5Cf","ver":1},"status":{"status":"write"}}},{"id":"1_1590332559.677626_216.1","section":"bucket","name":"ajxnhd-33","timestamp":"2020-05-24 15:02:39.677626433Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__TXIL7F0fo6f_ML5KkyR5Cf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332560.881764_217.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:40.881764475Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"status":{"status":"write"}}},{"id":"1_1590332561.064185_218.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:41.064185535Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332561.144892_219.1","section":"bucket","name":"ajxnhd-34","timestamp":"2020-05-24 15:02:41.144892141Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TnCet-EC2ujq4RjCMmtoi5b","ver":1},"status":{"status":"write"}}},{"id":"1_1590332561.397068_220.1","section":"bucket","name":"ajxnhd-34","timestamp":"2020-05-24 15:02:41.397068589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TnCet-EC2ujq4RjCMmtoi5b","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332562.761328_221.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:42.761328654Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"status":{"status":"write"}}},{"id":"1_1590332562.887604_222.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:42.887604521Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332562.947140_223.1","section":"bucket","name":"ajxnhd-35","timestamp":"2020-05-24 15:02:42.947140222Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4J9adjb6Qa0J6bXq6rh1nYz","ver":1},"status":{"status":"write"}}},{"id":"1_1590332563.062797_224.1","section":"bucket","name":"ajxnhd-35","timestamp":"2020-05-24 15:02:43.062797471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4J9adjb6Qa0J6bXq6rh1nYz","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332564.442927_225.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:44.442927388Z","data":{"read_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":2},"status":{"status":"write"}}},{"id":"1_1590332564.792134_226.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:44.792134507Z","data":{"read_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332566.229361_227.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:46.229361522Z","data":{"read_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":2},"status":{"status":"write"}}},{"id":"1_1590332566.526084_228.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:46.526084519Z","data":{"read_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332568.689676_229.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:48.689676599Z","data":{"read_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":2},"status":{"status":"write"}}},{"id":"1_1590332568.878454_230.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:48.878454727Z","data":{"read_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332570.049297_231.1","section":"bucket.instance","name":"ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","timestamp":"2020-05-24 15:02:50.049297529Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_b9k2iMA1NtKXNRMZLjKnuMK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332570.244663_232.1","section":"bucket.instance","name":"ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","timestamp":"2020-05-24 15:02:50.244663844Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_b9k2iMA1NtKXNRMZLjKnuMK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332570.372091_233.1","section":"bucket","name":"ajxnhd-36","timestamp":"2020-05-24 15:02:50.372091228Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7Jm22K8v7V5ENeX7fvvO6m3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332570.524216_234.1","section":"bucket","name":"ajxnhd-36","timestamp":"2020-05-24 15:02:50.524216691Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7Jm22K8v7V5ENeX7fvvO6m3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332570.961122_235.1","section":"bucket.instance","name":"ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","timestamp":"2020-05-24 15:02:50.961122188Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_24WmxNrnrGYWY6wlu6B-ieA","ver":1},"status":{"status":"write"}}},{"id":"1_1590332571.260025_236.1","section":"bucket.instance","name":"ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","timestamp":"2020-05-24 15:02:51.260025363Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_24WmxNrnrGYWY6wlu6B-ieA","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332571.358013_237.1","section":"bucket","name":"ajxnhd-37","timestamp":"2020-05-24 15:02:51.358013136Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sEb0Ve1wRSuEJNe56JIG4EI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332571.530066_238.1","section":"bucket","name":"ajxnhd-37","timestamp":"2020-05-24 15:02:51.530066130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sEb0Ve1wRSuEJNe56JIG4EI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332573.119556_239.1","section":"bucket.instance","name":"ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","timestamp":"2020-05-24 15:02:53.119556405Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7hfIFt8sICEW9pyMqr-MYV","ver":1},"status":{"status":"write"}}},{"id":"1_1590332573.260989_240.1","section":"bucket.instance","name":"ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","timestamp":"2020-05-24 15:02:53.260989054Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7hfIFt8sICEW9pyMqr-MYV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332573.335527_241.1","section":"bucket","name":"ajxnhd-38","timestamp":"2020-05-24 15:02:53.335527083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vNQMrHjRLeJ3QZbyKgH0O_6","ver":1},"status":{"status":"write"}}},{"id":"1_1590332573.500710_242.1","section":"bucket","name":"ajxnhd-38","timestamp":"2020-05-24 15:02:53.500710562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vNQMrHjRLeJ3QZbyKgH0O_6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.965748_243.1","section":"bucket.instance","name":"ajxnhd-39:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48","timestamp":"2020-05-24 15:03:11.965748944Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y58mPDPi7qXqG_CmkdmR6os","ver":1},"status":{"status":"write"}}},{"id":"1_1590332592.803259_244.1","section":"bucket.instance","name":"ajxnhd-39:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48","timestamp":"2020-05-24 15:03:12.803259188Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y58mPDPi7qXqG_CmkdmR6os","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.312415_245.1","section":"bucket","name":"ajxnhd-39","timestamp":"2020-05-24 15:03:13.312415629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UNwrQUdBwy_B1nsJ18BGrXu","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.915640_246.1","section":"bucket","name":"ajxnhd-39","timestamp":"2020-05-24 15:03:13.915640047Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UNwrQUdBwy_B1nsJ18BGrXu","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332595.681045_247.1","section":"bucket.instance","name":"ajxnhd-40:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49","timestamp":"2020-05-24 15:03:15.681045977Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cGbybOYRxqaUODITnQ64wX8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332595.965278_248.1","section":"bucket.instance","name":"ajxnhd-40:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49","timestamp":"2020-05-24 15:03:15.965278368Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cGbybOYRxqaUODITnQ64wX8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332596.121270_249.1","section":"bucket","name":"ajxnhd-40","timestamp":"2020-05-24 15:03:16.121270922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WSY76UVteF1dCj_5H4jHX5D","ver":1},"status":{"status":"write"}}},{"id":"1_1590332596.288693_250.1","section":"bucket","name":"ajxnhd-40","timestamp":"2020-05-24 15:03:16.288693632Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WSY76UVteF1dCj_5H4jHX5D","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332597.688431_251.1","section":"bucket.instance","name":"ajxnhd-41:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50","timestamp":"2020-05-24 15:03:17.688431472Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AK-EzZL3ZkVBaaCNyKRG8cA","ver":1},"status":{"status":"write"}}},{"id":"1_1590332597.968361_252.1","section":"bucket.instance","name":"ajxnhd-41:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50","timestamp":"2020-05-24 15:03:17.968361011Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AK-EzZL3ZkVBaaCNyKRG8cA","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332598.088015_253.1","section":"bucket","name":"ajxnhd-41","timestamp":"2020-05-24 15:03:18.088015374Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gs-0jKiD8zj7WuELPIX45Fk","ver":1},"status":{"status":"write"}}},{"id":"1_1590332598.292976_254.1","section":"bucket","name":"ajxnhd-41","timestamp":"2020-05-24 15:03:18.292976652Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gs-0jKiD8zj7WuELPIX45Fk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328661.857046_44.1","section":"bucket.instance","name":"ajxnhd-1:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1","timestamp":"2020-05-24 13:57:41.857046576Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EI_f_0Ki9favYOjyGg-58P0","ver":1},"status":{"status":"write"}}},{"id":"1_1590328663.801409_45.1","section":"bucket.instance","name":"ajxnhd-1:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1","timestamp":"2020-05-24 13:57:43.801409385Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EI_f_0Ki9favYOjyGg-58P0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328665.095008_46.1","section":"bucket","name":"ajxnhd-1","timestamp":"2020-05-24 13:57:45.095008350Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZoWBta9CxnaXqf3Hnfk32TH","ver":1},"status":{"status":"write"}}},{"id":"1_1590328666.568066_47.1","section":"bucket","name":"ajxnhd-1","timestamp":"2020-05-24 13:57:46.568066655Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZoWBta9CxnaXqf3Hnfk32TH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328669.388156_48.1","section":"bucket.instance","name":"ajxnhd-2:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2","timestamp":"2020-05-24 13:57:49.388156020Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GLHHyqK2R6W2PTMOkPNrrlk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328671.962074_49.1","section":"bucket.instance","name":"ajxnhd-2:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2","timestamp":"2020-05-24 13:57:51.962074124Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GLHHyqK2R6W2PTMOkPNrrlk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328674.256018_50.1","section":"bucket","name":"ajxnhd-2","timestamp":"2020-05-24 13:57:54.256018718Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGN9Rv5tiGtmUXakupLtXPZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328676.357935_51.1","section":"bucket","name":"ajxnhd-2","timestamp":"2020-05-24 13:57:56.357935846Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGN9Rv5tiGtmUXakupLtXPZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328681.147077_52.1","section":"bucket.instance","name":"ajxnhd-3:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3","timestamp":"2020-05-24 13:58:01.147077733Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9TpTYOx5Q03ed_c9goFYJZM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328681.299764_53.1","section":"bucket.instance","name":"ajxnhd-3:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3","timestamp":"2020-05-24 13:58:01.299764826Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9TpTYOx5Q03ed_c9goFYJZM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328681.371143_54.1","section":"bucket","name":"ajxnhd-3","timestamp":"2020-05-24 13:58:01.371143798Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vPul70r_8Ep8FRLwG3vWyNZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328681.510269_55.1","section":"bucket","name":"ajxnhd-3","timestamp":"2020-05-24 13:58:01.510269305Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vPul70r_8Ep8FRLwG3vWyNZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328695.908700_56.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4","timestamp":"2020-05-24 13:58:15.908700393Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vLXsYr9SmfTZ3Fb3d_wnAd3","ver":1},"status":{"status":"write"}}},{"id":"1_1590328696.114303_57.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4","timestamp":"2020-05-24 13:58:16.114303048Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vLXsYr9SmfTZ3Fb3d_wnAd3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328696.194626_58.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:16.194626894Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Jg0IeWtO91dabgJ-_9OekxY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328696.347412_59.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:16.347412194Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Jg0IeWtO91dabgJ-_9OekxY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328697.502176_60.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5","timestamp":"2020-05-24 13:58:17.502176684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LqXlxfl8QRw7-uDVpsgnDcw","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.668291_61.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5","timestamp":"2020-05-24 13:58:17.668291982Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LqXlxfl8QRw7-uDVpsgnDcw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328697.761208_62.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:17.761208203Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_obyOfpY3MrTzY_e23jlE70I","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.902648_63.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:17.902648222Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_obyOfpY3MrTzY_e23jlE70I","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328699.527169_64.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6","timestamp":"2020-05-24 13:58:19.527169223Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_o4Ky02fbN6YWn_f_ggrN7C7","ver":1},"status":{"status":"write"}}},{"id":"1_1590328699.677825_65.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6","timestamp":"2020-05-24 13:58:19.677825821Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_o4Ky02fbN6YWn_f_ggrN7C7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328699.833869_66.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:19.833869190Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_enfzxiViqv_w3BWd10aOZ0d","ver":1},"status":{"status":"write"}}},{"id":"1_1590328700.029897_67.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:20.029897018Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_enfzxiViqv_w3BWd10aOZ0d","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328714.515118_70.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.515118726Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"status":{"status":"write"}}},{"id":"1_1590328714.719389_71.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.719389695Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328714.782348_72.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:34.782348779Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4HVX6Tlmf_oEsDt4DBAWClI","ver":1},"status":{"status":"write"}}},{"id":"1_1590328714.897108_73.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:34.897108137Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4HVX6Tlmf_oEsDt4DBAWClI","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328714.993989_74.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.993989083Z","data":{"read_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328715.172258_75.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:35.172258827Z","data":{"read_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328715.564378_76.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:35.564378963Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"status":{"status":"write"}}},{"id":"1_1590328715.749934_77.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:35.749934229Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328715.857497_78.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:35.857497048Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NJuC_NimR8vZNocPMx3wfsi","ver":1},"status":{"status":"write"}}},{"id":"1_1590328716.050040_79.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:36.050040815Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NJuC_NimR8vZNocPMx3wfsi","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328716.187601_80.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:36.187601735Z","data":{"read_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328716.382658_81.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:36.382658090Z","data":{"read_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328716.732842_82.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:36.732842955Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328717.017969_83.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.017969634Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328717.178040_84.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:37.178040274Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sCm2MGtfgkAIqwwkQk3Yw5q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328717.373287_85.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:37.373287652Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sCm2MGtfgkAIqwwkQk3Yw5q","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328717.510560_86.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.510560445Z","data":{"read_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328717.684607_87.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.684607007Z","data":{"read_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328718.317405_89.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.317405148Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"status":{"status":"write"}}},{"id":"1_1590328718.504832_90.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.504832890Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328718.571111_91.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:38.571111899Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e8rj40fpi3-JZbsFLxS45Xk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328718.700562_92.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:38.700562070Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e8rj40fpi3-JZbsFLxS45Xk","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328718.860296_93.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.860296964Z","data":{"read_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328719.136857_94.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:39.136857916Z","data":{"read_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328720.315474_95.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.315474784Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"status":{"status":"write"}}},{"id":"1_1590328720.499875_96.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.499875922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328720.583970_97.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:40.583970910Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xi3oOUty4bv3bHoncH0dIz3","ver":1},"status":{"status":"write"}}},{"id":"1_1590328720.753543_98.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:40.753543145Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xi3oOUty4bv3bHoncH0dIz3","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328720.994211_99.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.994211183Z","data":{"read_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328721.267286_100.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:41.267286430Z","data":{"read_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328722.062273_101.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.062273899Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328722.334638_102.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.334638645Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328722.416735_103.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:42.416735183Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_dVkfYwfTCvS5gmH63FscSTy","ver":1},"status":{"status":"write"}}},{"id":"1_1590328722.542035_104.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:42.542035756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_dVkfYwfTCvS5gmH63FscSTy","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328722.640315_105.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.640315136Z","data":{"read_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328722.943793_106.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.943793752Z","data":{"read_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328724.614660_108.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:44.614660660Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328724.809740_109.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:44.809740816Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328724.882230_110.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:44.882230120Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UlCm7CE1xo5AsxtoMJbTMz4","ver":1},"status":{"status":"write"}}},{"id":"1_1590328725.087079_111.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:45.087079418Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UlCm7CE1xo5AsxtoMJbTMz4","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328725.239107_112.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:45.239107841Z","data":{"read_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328725.441830_113.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:45.441830031Z","data":{"read_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328726.292849_114.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.292849280Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"status":{"status":"write"}}},{"id":"1_1590328726.443715_115.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.443715821Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328726.524327_116.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:46.524327926Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VBLmoFhcIYIVqpzreE0XnKi","ver":1},"status":{"status":"write"}}},{"id":"1_1590328726.674271_117.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:46.674271737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VBLmoFhcIYIVqpzreE0XnKi","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328726.787812_118.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.787812842Z","data":{"read_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328727.010860_119.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:47.010860196Z","data":{"read_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328727.922311_120.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:47.922311552Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328728.057327_121.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.057327847Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328728.160326_122.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:48.160326113Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u5I6Hkn6tF0zFwYrBt4lf2D","ver":1},"status":{"status":"write"}}},{"id":"1_1590328728.345721_123.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:48.345721203Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u5I6Hkn6tF0zFwYrBt4lf2D","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328728.436742_124.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.436742651Z","data":{"read_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328728.582516_125.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.582516268Z","data":{"read_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328744.824204_129.1","section":"bucket.instance","name":"ajxnhd-7:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16","timestamp":"2020-05-24 13:59:04.824204578Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_azYpBimd6nndafzvDBhV2e6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328745.009557_130.1","section":"bucket.instance","name":"ajxnhd-7:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16","timestamp":"2020-05-24 13:59:05.009557060Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_azYpBimd6nndafzvDBhV2e6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328745.089321_131.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:05.089321631Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328745.283838_132.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:05.283838511Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328746.557992_133.1","section":"bucket.instance","name":"ajxnhd-8:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17","timestamp":"2020-05-24 13:59:06.557992864Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iJutw-w353HxM0wQe0ahGW0","ver":1},"status":{"status":"write"}}},{"id":"1_1590328746.745740_134.1","section":"bucket.instance","name":"ajxnhd-8:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17","timestamp":"2020-05-24 13:59:06.745740588Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iJutw-w353HxM0wQe0ahGW0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328746.807024_135.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:06.807024179Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"status":{"status":"write"}}},{"id":"1_1590328747.035908_136.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:07.035908477Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328749.986118_137.1","section":"bucket.instance","name":"ajxnhd-9:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18","timestamp":"2020-05-24 13:59:09.986118602Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_k86zaCt0pVAFBXphA-HHAKO","ver":1},"status":{"status":"write"}}},{"id":"1_1590328750.203959_138.1","section":"bucket.instance","name":"ajxnhd-9:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18","timestamp":"2020-05-24 13:59:10.203959616Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_k86zaCt0pVAFBXphA-HHAKO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328750.291133_139.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:10.291133261Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"status":{"status":"write"}}},{"id":"1_1590328750.444965_140.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:10.444965711Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328764.224998_141.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:24.224998965Z","data":{"read_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328764.442285_142.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:24.442285080Z","data":{"read_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328765.812508_143.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:25.812508979Z","data":{"read_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328765.976327_144.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:25.976327786Z","data":{"read_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328767.020248_145.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:27.020248606Z","data":{"read_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328767.216616_146.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:27.216616731Z","data":{"read_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":2},"status":{"status":"complete"}}}] 2020-05-24T15:06:49.939 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:49.940 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:49.949 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 114a3cfc-32a6-4ae9-bab5-baac4f59a79e --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:53.022 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":"1_1590328567.776739_38.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:07.776739104Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ui15Zsog5OeOBfQLFnhWmAY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328568.276896_39.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:08.276896545Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ui15Zsog5OeOBfQLFnhWmAY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328780.710920_36.1","section":"bucket.instance","name":"ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","timestamp":"2020-05-24 13:59:40.710920367Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_1x1ZZjoXeMNzJoUjkg5bWmM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328780.907463_37.1","section":"bucket.instance","name":"ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","timestamp":"2020-05-24 13:59:40.907463503Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_1x1ZZjoXeMNzJoUjkg5bWmM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328780.992516_38.1","section":"bucket","name":"ajxnhd-10","timestamp":"2020-05-24 13:59:40.992516918Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_F0eV8p4gy4VVoOyBszXK9Uy","ver":1},"status":{"status":"write"}}},{"id":"1_1590328781.173823_39.1","section":"bucket","name":"ajxnhd-10","timestamp":"2020-05-24 13:59:41.173823390Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_F0eV8p4gy4VVoOyBszXK9Uy","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328781.786763_40.1","section":"bucket.instance","name":"ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","timestamp":"2020-05-24 13:59:41.786763813Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yAfrDCGApzmfUfpLV32eRYD","ver":1},"status":{"status":"write"}}},{"id":"1_1590328782.049055_41.1","section":"bucket.instance","name":"ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","timestamp":"2020-05-24 13:59:42.049055331Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yAfrDCGApzmfUfpLV32eRYD","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328782.155023_42.1","section":"bucket","name":"ajxnhd-11","timestamp":"2020-05-24 13:59:42.155023264Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6JZ7TwZSk3ER0WTRkMdpYxK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328782.443391_43.1","section":"bucket","name":"ajxnhd-11","timestamp":"2020-05-24 13:59:42.443391351Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6JZ7TwZSk3ER0WTRkMdpYxK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.411331_44.1","section":"bucket.instance","name":"ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","timestamp":"2020-05-24 13:59:43.411331039Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5HqOnT066cNskmDRsRqZ-dE","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.597981_45.1","section":"bucket.instance","name":"ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","timestamp":"2020-05-24 13:59:43.597981263Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5HqOnT066cNskmDRsRqZ-dE","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.683466_46.1","section":"bucket","name":"ajxnhd-12","timestamp":"2020-05-24 13:59:43.683466272Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_z0ez1Ljq8OH081qJv8Q7CL6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.837755_47.1","section":"bucket","name":"ajxnhd-12","timestamp":"2020-05-24 13:59:43.837755756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_z0ez1Ljq8OH081qJv8Q7CL6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329320.125728_54.1","section":"bucket.instance","name":"ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","timestamp":"2020-05-24 14:08:40.125728151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NEqYrxBauZKMtGHHXFkIZCq","ver":1},"status":{"status":"write"}}},{"id":"1_1590329320.290517_55.1","section":"bucket.instance","name":"ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","timestamp":"2020-05-24 14:08:40.290517017Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NEqYrxBauZKMtGHHXFkIZCq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329320.400997_56.1","section":"bucket","name":"ajxnhd-13","timestamp":"2020-05-24 14:08:40.400997847Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eweXCos3PdlUyYekfNFjSHH","ver":1},"status":{"status":"write"}}},{"id":"1_1590329320.572773_57.1","section":"bucket","name":"ajxnhd-13","timestamp":"2020-05-24 14:08:40.572773700Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eweXCos3PdlUyYekfNFjSHH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329322.157703_58.1","section":"bucket.instance","name":"ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","timestamp":"2020-05-24 14:08:42.157703103Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BmyQDJAOboCFiJ45lLtofKE","ver":1},"status":{"status":"write"}}},{"id":"1_1590329322.303444_59.1","section":"bucket.instance","name":"ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","timestamp":"2020-05-24 14:08:42.303444730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BmyQDJAOboCFiJ45lLtofKE","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329322.485950_60.1","section":"bucket","name":"ajxnhd-14","timestamp":"2020-05-24 14:08:42.485950917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_quzz_j7V4et7RpGVfsZ8DI3","ver":1},"status":{"status":"write"}}},{"id":"1_1590329322.753097_61.1","section":"bucket","name":"ajxnhd-14","timestamp":"2020-05-24 14:08:42.753097194Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_quzz_j7V4et7RpGVfsZ8DI3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329323.912546_62.1","section":"bucket.instance","name":"ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","timestamp":"2020-05-24 14:08:43.912546421Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ljytbx-c-ioF_jOyI19KhIn","ver":1},"status":{"status":"write"}}},{"id":"1_1590329324.055027_63.1","section":"bucket.instance","name":"ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","timestamp":"2020-05-24 14:08:44.055027596Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ljytbx-c-ioF_jOyI19KhIn","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329324.140936_64.1","section":"bucket","name":"ajxnhd-15","timestamp":"2020-05-24 14:08:44.140936259Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__NTNWGbez_pXSCfSDkXBqXJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590329324.279876_65.1","section":"bucket","name":"ajxnhd-15","timestamp":"2020-05-24 14:08:44.279876745Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__NTNWGbez_pXSCfSDkXBqXJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329851.002187_72.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:31.002187986Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"status":{"status":"write"}}},{"id":"1_1590329851.192404_73.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:31.192404499Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329851.255970_74.1","section":"bucket","name":"ajxnhd-16","timestamp":"2020-05-24 14:17:31.255970913Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_n5pa9q08aZnglX0ZBX2MI72","ver":1},"status":{"status":"write"}}},{"id":"1_1590329851.450578_75.1","section":"bucket","name":"ajxnhd-16","timestamp":"2020-05-24 14:17:31.450578151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_n5pa9q08aZnglX0ZBX2MI72","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329852.156682_76.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:32.156682053Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"status":{"status":"write"}}},{"id":"1_1590329852.314606_77.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:32.314606785Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329852.385946_78.1","section":"bucket","name":"ajxnhd-17","timestamp":"2020-05-24 14:17:32.385946764Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2yVMkwTR0dwxoaxsJDfspmT","ver":1},"status":{"status":"write"}}},{"id":"1_1590329852.537860_79.1","section":"bucket","name":"ajxnhd-17","timestamp":"2020-05-24 14:17:32.537860267Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2yVMkwTR0dwxoaxsJDfspmT","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329853.921024_80.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:33.921024364Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"status":{"status":"write"}}},{"id":"1_1590329854.087205_81.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:34.087205756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329854.204871_82.1","section":"bucket","name":"ajxnhd-18","timestamp":"2020-05-24 14:17:34.204871979Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBs2gR836NQmjWKNfHbjG8K","ver":1},"status":{"status":"write"}}},{"id":"1_1590329854.445256_83.1","section":"bucket","name":"ajxnhd-18","timestamp":"2020-05-24 14:17:34.445256737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBs2gR836NQmjWKNfHbjG8K","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329855.079839_84.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:35.079839696Z","data":{"read_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":2},"status":{"status":"write"}}},{"id":"1_1590329855.330800_85.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:35.330800723Z","data":{"read_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329856.118614_86.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:36.118614343Z","data":{"read_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":2},"status":{"status":"write"}}},{"id":"1_1590329856.475914_87.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:36.475914391Z","data":{"read_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329857.623824_88.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:37.623824536Z","data":{"read_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":2},"status":{"status":"write"}}},{"id":"1_1590329857.827328_89.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:37.827328629Z","data":{"read_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330379.734826_96.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:19.734826079Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"status":{"status":"write"}}},{"id":"1_1590330379.889098_97.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:19.889098852Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330380.055101_98.1","section":"bucket","name":"ajxnhd-19","timestamp":"2020-05-24 14:26:20.055101601Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VogH8d_v03-pKxveKvcQRpR","ver":1},"status":{"status":"write"}}},{"id":"1_1590330380.186271_99.1","section":"bucket","name":"ajxnhd-19","timestamp":"2020-05-24 14:26:20.186271981Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VogH8d_v03-pKxveKvcQRpR","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330380.342253_100.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:20.342253953Z","data":{"read_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":2},"status":{"status":"write"}}},{"id":"1_1590330380.583029_101.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:20.583029777Z","data":{"read_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330903.436321_102.1","section":"bucket.instance","name":"ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","timestamp":"2020-05-24 14:35:03.436321562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gAFo5R_jIeCyFq2LSsNo17_","ver":1},"status":{"status":"write"}}},{"id":"1_1590330903.717682_103.1","section":"bucket.instance","name":"ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","timestamp":"2020-05-24 14:35:03.717682381Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gAFo5R_jIeCyFq2LSsNo17_","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330903.810200_104.1","section":"bucket","name":"ajxnhd-20","timestamp":"2020-05-24 14:35:03.810200796Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zOYKFnexj8DFjS8jtUKG4DJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590330903.938053_105.1","section":"bucket","name":"ajxnhd-20","timestamp":"2020-05-24 14:35:03.938053252Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zOYKFnexj8DFjS8jtUKG4DJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331429.252015_120.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:49.252015956Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"status":{"status":"write"}}},{"id":"1_1590331429.406239_121.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:49.406239502Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331429.478659_122.1","section":"bucket","name":"ajxnhd-21","timestamp":"2020-05-24 14:43:49.478659851Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LhHMDRFw1MbeqVRBXiLVGqV","ver":1},"status":{"status":"write"}}},{"id":"1_1590331429.775227_123.1","section":"bucket","name":"ajxnhd-21","timestamp":"2020-05-24 14:43:49.775227278Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LhHMDRFw1MbeqVRBXiLVGqV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331430.455137_124.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:50.455137998Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"status":{"status":"write"}}},{"id":"1_1590331430.680723_125.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:50.680723110Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331430.776858_126.1","section":"bucket","name":"ajxnhd-22","timestamp":"2020-05-24 14:43:50.776858522Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zS0Qv2MgKEFW14qiK5JnkWJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590331430.897169_127.1","section":"bucket","name":"ajxnhd-22","timestamp":"2020-05-24 14:43:50.897169843Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zS0Qv2MgKEFW14qiK5JnkWJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331432.003953_128.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:52.003953530Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"status":{"status":"write"}}},{"id":"1_1590331432.142214_129.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:52.142214510Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331432.205496_130.1","section":"bucket","name":"ajxnhd-23","timestamp":"2020-05-24 14:43:52.205496460Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yrznneKnovuxIXNnXnO0uLI","ver":1},"status":{"status":"write"}}},{"id":"1_1590331432.337983_131.1","section":"bucket","name":"ajxnhd-23","timestamp":"2020-05-24 14:43:52.337983599Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yrznneKnovuxIXNnXnO0uLI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331433.714200_132.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:53.714200278Z","data":{"read_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":2},"status":{"status":"write"}}},{"id":"1_1590331433.948922_133.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:53.948922752Z","data":{"read_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331434.894781_134.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:54.894781681Z","data":{"read_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":2},"status":{"status":"write"}}},{"id":"1_1590331435.198978_135.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:55.198978175Z","data":{"read_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331435.928156_136.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:55.928156482Z","data":{"read_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":2},"status":{"status":"write"}}},{"id":"1_1590331436.244684_137.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:56.244684179Z","data":{"read_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331983.049122_144.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:03.049122099Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"status":{"status":"write"}}},{"id":"1_1590331983.197037_145.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:03.197037858Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331983.266790_146.1","section":"bucket","name":"ajxnhd-24","timestamp":"2020-05-24 14:53:03.266790936Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uIlRzMpJio6QSnre8AMGFLm","ver":1},"status":{"status":"write"}}},{"id":"1_1590331983.424889_147.1","section":"bucket","name":"ajxnhd-24","timestamp":"2020-05-24 14:53:03.424889595Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uIlRzMpJio6QSnre8AMGFLm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331983.865257_148.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:03.865257449Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"status":{"status":"write"}}},{"id":"1_1590331984.122968_149.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:04.122968508Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331984.251929_150.1","section":"bucket","name":"ajxnhd-25","timestamp":"2020-05-24 14:53:04.251929448Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uQJr9MjpeQurPAVQIXSW_JM","ver":1},"status":{"status":"write"}}},{"id":"1_1590331984.505736_151.1","section":"bucket","name":"ajxnhd-25","timestamp":"2020-05-24 14:53:04.505736130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uQJr9MjpeQurPAVQIXSW_JM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331986.438990_152.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:06.438990629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"status":{"status":"write"}}},{"id":"1_1590331986.585637_153.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:06.585637806Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331986.647556_154.1","section":"bucket","name":"ajxnhd-26","timestamp":"2020-05-24 14:53:06.647556290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Al_0YQq_jeAkOhBQMDkf2Z7","ver":1},"status":{"status":"write"}}},{"id":"1_1590331986.814186_155.1","section":"bucket","name":"ajxnhd-26","timestamp":"2020-05-24 14:53:06.814186094Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Al_0YQq_jeAkOhBQMDkf2Z7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331988.439385_156.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.439385111Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"status":{"status":"write"}}},{"id":"1_1590331988.794207_157.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.794207447Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331988.879023_158.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.879023435Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":3},"status":{"status":"write"}}},{"id":"1_1590331989.060960_159.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:09.060960790Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":3},"status":{"status":"complete"}}},{"id":"1_1590331989.527880_160.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:09.527880038Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"status":{"status":"write"}}},{"id":"1_1590331989.813285_161.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:09.813285672Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331990.123113_162.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:10.123113736Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":3},"status":{"status":"write"}}},{"id":"1_1590331990.429901_163.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:10.429901443Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":3},"status":{"status":"complete"}}},{"id":"1_1590331991.738750_164.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:11.738750891Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"status":{"status":"write"}}},{"id":"1_1590331991.926433_165.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:11.926433769Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331992.233007_166.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:12.233007240Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":3},"status":{"status":"write"}}},{"id":"1_1590331992.462010_167.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:12.462010498Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332542.631802_177.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:22.631802311Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"status":{"status":"write"}}},{"id":"1_1590332542.830206_178.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:22.830206865Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332542.936016_179.1","section":"bucket","name":"ajxnhd-27","timestamp":"2020-05-24 15:02:22.936016582Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6-ZQ1R4_Nnl5GdcjPwHQgoI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332543.103013_180.1","section":"bucket","name":"ajxnhd-27","timestamp":"2020-05-24 15:02:23.103013361Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6-ZQ1R4_Nnl5GdcjPwHQgoI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332543.775583_181.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:23.775583592Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"status":{"status":"write"}}},{"id":"1_1590332544.023476_182.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:24.023476172Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332544.120691_183.1","section":"bucket","name":"ajxnhd-28","timestamp":"2020-05-24 15:02:24.120691142Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_S_0X21LYEXlkSLS9-UznAcI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332544.302282_184.1","section":"bucket","name":"ajxnhd-28","timestamp":"2020-05-24 15:02:24.302282945Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_S_0X21LYEXlkSLS9-UznAcI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332545.404781_185.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:25.404781963Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332545.585733_186.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:25.585733302Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332545.654256_187.1","section":"bucket","name":"ajxnhd-29","timestamp":"2020-05-24 15:02:25.654256391Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vqZzPK8m08iLaKomZNlavH0","ver":1},"status":{"status":"write"}}},{"id":"1_1590332545.771078_188.1","section":"bucket","name":"ajxnhd-29","timestamp":"2020-05-24 15:02:25.771078802Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vqZzPK8m08iLaKomZNlavH0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332546.555221_189.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:26.555221665Z","data":{"read_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":2},"status":{"status":"write"}}},{"id":"1_1590332546.717101_190.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:26.717101024Z","data":{"read_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332547.859104_191.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:27.859104280Z","data":{"read_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":2},"status":{"status":"write"}}},{"id":"1_1590332548.198444_192.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:28.198444891Z","data":{"read_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332548.675933_193.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:28.675933514Z","data":{"read_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":2},"status":{"status":"write"}}},{"id":"1_1590332549.028549_194.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:29.028549384Z","data":{"read_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332549.967216_195.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:29.967216514Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"status":{"status":"write"}}},{"id":"1_1590332550.115459_196.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:30.115459573Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332550.244077_197.1","section":"bucket","name":"ajxnhd-30","timestamp":"2020-05-24 15:02:30.244077484Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P61wwDe_JM5YuqmV6LehKBK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332550.456855_198.1","section":"bucket","name":"ajxnhd-30","timestamp":"2020-05-24 15:02:30.456855328Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P61wwDe_JM5YuqmV6LehKBK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332551.822124_199.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:31.822124501Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"status":{"status":"write"}}},{"id":"1_1590332552.065161_200.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:32.065161497Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332552.144138_201.1","section":"bucket","name":"ajxnhd-31","timestamp":"2020-05-24 15:02:32.144138917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_guBR1c3-SAHv8LKMh2QLb7A","ver":1},"status":{"status":"write"}}},{"id":"1_1590332552.260354_202.1","section":"bucket","name":"ajxnhd-31","timestamp":"2020-05-24 15:02:32.260354003Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_guBR1c3-SAHv8LKMh2QLb7A","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332553.270432_203.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:33.270432895Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"status":{"status":"write"}}},{"id":"1_1590332553.415925_204.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:33.415925128Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332553.486668_205.1","section":"bucket","name":"ajxnhd-32","timestamp":"2020-05-24 15:02:33.486668591Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_s1Byxm5NDVFw_jUWDF_FjgS","ver":1},"status":{"status":"write"}}},{"id":"1_1590332553.660263_206.1","section":"bucket","name":"ajxnhd-32","timestamp":"2020-05-24 15:02:33.660263701Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_s1Byxm5NDVFw_jUWDF_FjgS","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332554.995884_207.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:34.995884736Z","data":{"read_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":2},"status":{"status":"write"}}},{"id":"1_1590332555.358899_208.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:35.358899184Z","data":{"read_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332556.426757_209.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:36.426757678Z","data":{"read_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":2},"status":{"status":"write"}}},{"id":"1_1590332556.687130_210.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:36.687130128Z","data":{"read_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332557.695414_211.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:37.695414905Z","data":{"read_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":2},"status":{"status":"write"}}},{"id":"1_1590332557.981422_212.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:37.981422719Z","data":{"read_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332559.305198_213.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:39.305198079Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"status":{"status":"write"}}},{"id":"1_1590332559.482651_214.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:39.482651412Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332559.540073_215.1","section":"bucket","name":"ajxnhd-33","timestamp":"2020-05-24 15:02:39.540073474Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__TXIL7F0fo6f_ML5KkyR5Cf","ver":1},"status":{"status":"write"}}},{"id":"1_1590332559.677626_216.1","section":"bucket","name":"ajxnhd-33","timestamp":"2020-05-24 15:02:39.677626433Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__TXIL7F0fo6f_ML5KkyR5Cf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332560.881764_217.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:40.881764475Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"status":{"status":"write"}}},{"id":"1_1590332561.064185_218.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:41.064185535Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332561.144892_219.1","section":"bucket","name":"ajxnhd-34","timestamp":"2020-05-24 15:02:41.144892141Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TnCet-EC2ujq4RjCMmtoi5b","ver":1},"status":{"status":"write"}}},{"id":"1_1590332561.397068_220.1","section":"bucket","name":"ajxnhd-34","timestamp":"2020-05-24 15:02:41.397068589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TnCet-EC2ujq4RjCMmtoi5b","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332562.761328_221.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:42.761328654Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"status":{"status":"write"}}},{"id":"1_1590332562.887604_222.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:42.887604521Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332562.947140_223.1","section":"bucket","name":"ajxnhd-35","timestamp":"2020-05-24 15:02:42.947140222Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4J9adjb6Qa0J6bXq6rh1nYz","ver":1},"status":{"status":"write"}}},{"id":"1_1590332563.062797_224.1","section":"bucket","name":"ajxnhd-35","timestamp":"2020-05-24 15:02:43.062797471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4J9adjb6Qa0J6bXq6rh1nYz","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332564.442927_225.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:44.442927388Z","data":{"read_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":2},"status":{"status":"write"}}},{"id":"1_1590332564.792134_226.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:44.792134507Z","data":{"read_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332566.229361_227.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:46.229361522Z","data":{"read_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":2},"status":{"status":"write"}}},{"id":"1_1590332566.526084_228.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:46.526084519Z","data":{"read_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332568.689676_229.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:48.689676599Z","data":{"read_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":2},"status":{"status":"write"}}},{"id":"1_1590332568.878454_230.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:48.878454727Z","data":{"read_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332570.049297_231.1","section":"bucket.instance","name":"ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","timestamp":"2020-05-24 15:02:50.049297529Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_b9k2iMA1NtKXNRMZLjKnuMK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332570.244663_232.1","section":"bucket.instance","name":"ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","timestamp":"2020-05-24 15:02:50.244663844Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_b9k2iMA1NtKXNRMZLjKnuMK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332570.372091_233.1","section":"bucket","name":"ajxnhd-36","timestamp":"2020-05-24 15:02:50.372091228Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7Jm22K8v7V5ENeX7fvvO6m3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332570.524216_234.1","section":"bucket","name":"ajxnhd-36","timestamp":"2020-05-24 15:02:50.524216691Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7Jm22K8v7V5ENeX7fvvO6m3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332570.961122_235.1","section":"bucket.instance","name":"ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","timestamp":"2020-05-24 15:02:50.961122188Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_24WmxNrnrGYWY6wlu6B-ieA","ver":1},"status":{"status":"write"}}},{"id":"1_1590332571.260025_236.1","section":"bucket.instance","name":"ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","timestamp":"2020-05-24 15:02:51.260025363Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_24WmxNrnrGYWY6wlu6B-ieA","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332571.358013_237.1","section":"bucket","name":"ajxnhd-37","timestamp":"2020-05-24 15:02:51.358013136Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sEb0Ve1wRSuEJNe56JIG4EI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332571.530066_238.1","section":"bucket","name":"ajxnhd-37","timestamp":"2020-05-24 15:02:51.530066130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sEb0Ve1wRSuEJNe56JIG4EI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332573.119556_239.1","section":"bucket.instance","name":"ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","timestamp":"2020-05-24 15:02:53.119556405Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7hfIFt8sICEW9pyMqr-MYV","ver":1},"status":{"status":"write"}}},{"id":"1_1590332573.260989_240.1","section":"bucket.instance","name":"ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","timestamp":"2020-05-24 15:02:53.260989054Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7hfIFt8sICEW9pyMqr-MYV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332573.335527_241.1","section":"bucket","name":"ajxnhd-38","timestamp":"2020-05-24 15:02:53.335527083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vNQMrHjRLeJ3QZbyKgH0O_6","ver":1},"status":{"status":"write"}}},{"id":"1_1590332573.500710_242.1","section":"bucket","name":"ajxnhd-38","timestamp":"2020-05-24 15:02:53.500710562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vNQMrHjRLeJ3QZbyKgH0O_6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.965748_243.1","section":"bucket.instance","name":"ajxnhd-39:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48","timestamp":"2020-05-24 15:03:11.965748944Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y58mPDPi7qXqG_CmkdmR6os","ver":1},"status":{"status":"write"}}},{"id":"1_1590332592.803259_244.1","section":"bucket.instance","name":"ajxnhd-39:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48","timestamp":"2020-05-24 15:03:12.803259188Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y58mPDPi7qXqG_CmkdmR6os","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.312415_245.1","section":"bucket","name":"ajxnhd-39","timestamp":"2020-05-24 15:03:13.312415629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UNwrQUdBwy_B1nsJ18BGrXu","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.915640_246.1","section":"bucket","name":"ajxnhd-39","timestamp":"2020-05-24 15:03:13.915640047Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UNwrQUdBwy_B1nsJ18BGrXu","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332595.681045_247.1","section":"bucket.instance","name":"ajxnhd-40:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49","timestamp":"2020-05-24 15:03:15.681045977Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cGbybOYRxqaUODITnQ64wX8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332595.965278_248.1","section":"bucket.instance","name":"ajxnhd-40:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49","timestamp":"2020-05-24 15:03:15.965278368Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cGbybOYRxqaUODITnQ64wX8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332596.121270_249.1","section":"bucket","name":"ajxnhd-40","timestamp":"2020-05-24 15:03:16.121270922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WSY76UVteF1dCj_5H4jHX5D","ver":1},"status":{"status":"write"}}},{"id":"1_1590332596.288693_250.1","section":"bucket","name":"ajxnhd-40","timestamp":"2020-05-24 15:03:16.288693632Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WSY76UVteF1dCj_5H4jHX5D","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332597.688431_251.1","section":"bucket.instance","name":"ajxnhd-41:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50","timestamp":"2020-05-24 15:03:17.688431472Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AK-EzZL3ZkVBaaCNyKRG8cA","ver":1},"status":{"status":"write"}}},{"id":"1_1590332597.968361_252.1","section":"bucket.instance","name":"ajxnhd-41:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50","timestamp":"2020-05-24 15:03:17.968361011Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AK-EzZL3ZkVBaaCNyKRG8cA","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332598.088015_253.1","section":"bucket","name":"ajxnhd-41","timestamp":"2020-05-24 15:03:18.088015374Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gs-0jKiD8zj7WuELPIX45Fk","ver":1},"status":{"status":"write"}}},{"id":"1_1590332598.292976_254.1","section":"bucket","name":"ajxnhd-41","timestamp":"2020-05-24 15:03:18.292976652Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gs-0jKiD8zj7WuELPIX45Fk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328661.857046_44.1","section":"bucket.instance","name":"ajxnhd-1:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1","timestamp":"2020-05-24 13:57:41.857046576Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EI_f_0Ki9favYOjyGg-58P0","ver":1},"status":{"status":"write"}}},{"id":"1_1590328663.801409_45.1","section":"bucket.instance","name":"ajxnhd-1:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1","timestamp":"2020-05-24 13:57:43.801409385Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EI_f_0Ki9favYOjyGg-58P0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328665.095008_46.1","section":"bucket","name":"ajxnhd-1","timestamp":"2020-05-24 13:57:45.095008350Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZoWBta9CxnaXqf3Hnfk32TH","ver":1},"status":{"status":"write"}}},{"id":"1_1590328666.568066_47.1","section":"bucket","name":"ajxnhd-1","timestamp":"2020-05-24 13:57:46.568066655Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZoWBta9CxnaXqf3Hnfk32TH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328669.388156_48.1","section":"bucket.instance","name":"ajxnhd-2:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2","timestamp":"2020-05-24 13:57:49.388156020Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GLHHyqK2R6W2PTMOkPNrrlk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328671.962074_49.1","section":"bucket.instance","name":"ajxnhd-2:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2","timestamp":"2020-05-24 13:57:51.962074124Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GLHHyqK2R6W2PTMOkPNrrlk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328674.256018_50.1","section":"bucket","name":"ajxnhd-2","timestamp":"2020-05-24 13:57:54.256018718Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGN9Rv5tiGtmUXakupLtXPZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328676.357935_51.1","section":"bucket","name":"ajxnhd-2","timestamp":"2020-05-24 13:57:56.357935846Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGN9Rv5tiGtmUXakupLtXPZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328681.147077_52.1","section":"bucket.instance","name":"ajxnhd-3:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3","timestamp":"2020-05-24 13:58:01.147077733Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9TpTYOx5Q03ed_c9goFYJZM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328681.299764_53.1","section":"bucket.instance","name":"ajxnhd-3:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3","timestamp":"2020-05-24 13:58:01.299764826Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9TpTYOx5Q03ed_c9goFYJZM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328681.371143_54.1","section":"bucket","name":"ajxnhd-3","timestamp":"2020-05-24 13:58:01.371143798Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vPul70r_8Ep8FRLwG3vWyNZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328681.510269_55.1","section":"bucket","name":"ajxnhd-3","timestamp":"2020-05-24 13:58:01.510269305Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vPul70r_8Ep8FRLwG3vWyNZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328695.908700_56.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4","timestamp":"2020-05-24 13:58:15.908700393Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vLXsYr9SmfTZ3Fb3d_wnAd3","ver":1},"status":{"status":"write"}}},{"id":"1_1590328696.114303_57.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4","timestamp":"2020-05-24 13:58:16.114303048Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vLXsYr9SmfTZ3Fb3d_wnAd3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328696.194626_58.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:16.194626894Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Jg0IeWtO91dabgJ-_9OekxY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328696.347412_59.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:16.347412194Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Jg0IeWtO91dabgJ-_9OekxY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328697.502176_60.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5","timestamp":"2020-05-24 13:58:17.502176684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LqXlxfl8QRw7-uDVpsgnDcw","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.668291_61.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5","timestamp":"2020-05-24 13:58:17.668291982Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LqXlxfl8QRw7-uDVpsgnDcw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328697.761208_62.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:17.761208203Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_obyOfpY3MrTzY_e23jlE70I","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.902648_63.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:17.902648222Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_obyOfpY3MrTzY_e23jlE70I","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328699.527169_64.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6","timestamp":"2020-05-24 13:58:19.527169223Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_o4Ky02fbN6YWn_f_ggrN7C7","ver":1},"status":{"status":"write"}}},{"id":"1_1590328699.677825_65.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6","timestamp":"2020-05-24 13:58:19.677825821Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_o4Ky02fbN6YWn_f_ggrN7C7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328699.833869_66.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:19.833869190Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_enfzxiViqv_w3BWd10aOZ0d","ver":1},"status":{"status":"write"}}},{"id":"1_1590328700.029897_67.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:20.029897018Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_enfzxiViqv_w3BWd10aOZ0d","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328714.515118_70.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.515118726Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"status":{"status":"write"}}},{"id":"1_1590328714.719389_71.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.719389695Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328714.782348_72.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:34.782348779Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4HVX6Tlmf_oEsDt4DBAWClI","ver":1},"status":{"status":"write"}}},{"id":"1_1590328714.897108_73.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:34.897108137Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4HVX6Tlmf_oEsDt4DBAWClI","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328714.993989_74.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.993989083Z","data":{"read_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328715.172258_75.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:35.172258827Z","data":{"read_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328715.564378_76.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:35.564378963Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"status":{"status":"write"}}},{"id":"1_1590328715.749934_77.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:35.749934229Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328715.857497_78.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:35.857497048Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NJuC_NimR8vZNocPMx3wfsi","ver":1},"status":{"status":"write"}}},{"id":"1_1590328716.050040_79.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:36.050040815Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NJuC_NimR8vZNocPMx3wfsi","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328716.187601_80.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:36.187601735Z","data":{"read_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328716.382658_81.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:36.382658090Z","data":{"read_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328716.732842_82.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:36.732842955Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328717.017969_83.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.017969634Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328717.178040_84.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:37.178040274Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sCm2MGtfgkAIqwwkQk3Yw5q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328717.373287_85.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:37.373287652Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sCm2MGtfgkAIqwwkQk3Yw5q","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328717.510560_86.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.510560445Z","data":{"read_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328717.684607_87.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.684607007Z","data":{"read_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328718.317405_89.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.317405148Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"status":{"status":"write"}}},{"id":"1_1590328718.504832_90.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.504832890Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328718.571111_91.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:38.571111899Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e8rj40fpi3-JZbsFLxS45Xk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328718.700562_92.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:38.700562070Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e8rj40fpi3-JZbsFLxS45Xk","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328718.860296_93.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.860296964Z","data":{"read_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328719.136857_94.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:39.136857916Z","data":{"read_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328720.315474_95.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.315474784Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"status":{"status":"write"}}},{"id":"1_1590328720.499875_96.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.499875922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328720.583970_97.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:40.583970910Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xi3oOUty4bv3bHoncH0dIz3","ver":1},"status":{"status":"write"}}},{"id":"1_1590328720.753543_98.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:40.753543145Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xi3oOUty4bv3bHoncH0dIz3","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328720.994211_99.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.994211183Z","data":{"read_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328721.267286_100.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:41.267286430Z","data":{"read_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328722.062273_101.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.062273899Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328722.334638_102.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.334638645Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328722.416735_103.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:42.416735183Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_dVkfYwfTCvS5gmH63FscSTy","ver":1},"status":{"status":"write"}}},{"id":"1_1590328722.542035_104.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:42.542035756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_dVkfYwfTCvS5gmH63FscSTy","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328722.640315_105.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.640315136Z","data":{"read_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328722.943793_106.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.943793752Z","data":{"read_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328724.614660_108.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:44.614660660Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328724.809740_109.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:44.809740816Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328724.882230_110.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:44.882230120Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UlCm7CE1xo5AsxtoMJbTMz4","ver":1},"status":{"status":"write"}}},{"id":"1_1590328725.087079_111.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:45.087079418Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UlCm7CE1xo5AsxtoMJbTMz4","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328725.239107_112.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:45.239107841Z","data":{"read_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328725.441830_113.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:45.441830031Z","data":{"read_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328726.292849_114.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.292849280Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"status":{"status":"write"}}},{"id":"1_1590328726.443715_115.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.443715821Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328726.524327_116.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:46.524327926Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VBLmoFhcIYIVqpzreE0XnKi","ver":1},"status":{"status":"write"}}},{"id":"1_1590328726.674271_117.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:46.674271737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VBLmoFhcIYIVqpzreE0XnKi","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328726.787812_118.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.787812842Z","data":{"read_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328727.010860_119.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:47.010860196Z","data":{"read_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328727.922311_120.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:47.922311552Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328728.057327_121.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.057327847Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328728.160326_122.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:48.160326113Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u5I6Hkn6tF0zFwYrBt4lf2D","ver":1},"status":{"status":"write"}}},{"id":"1_1590328728.345721_123.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:48.345721203Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u5I6Hkn6tF0zFwYrBt4lf2D","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328728.436742_124.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.436742651Z","data":{"read_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328728.582516_125.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.582516268Z","data":{"read_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328744.824204_129.1","section":"bucket.instance","name":"ajxnhd-7:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16","timestamp":"2020-05-24 13:59:04.824204578Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_azYpBimd6nndafzvDBhV2e6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328745.009557_130.1","section":"bucket.instance","name":"ajxnhd-7:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16","timestamp":"2020-05-24 13:59:05.009557060Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_azYpBimd6nndafzvDBhV2e6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328745.089321_131.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:05.089321631Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328745.283838_132.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:05.283838511Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328746.557992_133.1","section":"bucket.instance","name":"ajxnhd-8:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17","timestamp":"2020-05-24 13:59:06.557992864Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iJutw-w353HxM0wQe0ahGW0","ver":1},"status":{"status":"write"}}},{"id":"1_1590328746.745740_134.1","section":"bucket.instance","name":"ajxnhd-8:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17","timestamp":"2020-05-24 13:59:06.745740588Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iJutw-w353HxM0wQe0ahGW0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328746.807024_135.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:06.807024179Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"status":{"status":"write"}}},{"id":"1_1590328747.035908_136.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:07.035908477Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328749.986118_137.1","section":"bucket.instance","name":"ajxnhd-9:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18","timestamp":"2020-05-24 13:59:09.986118602Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_k86zaCt0pVAFBXphA-HHAKO","ver":1},"status":{"status":"write"}}},{"id":"1_1590328750.203959_138.1","section":"bucket.instance","name":"ajxnhd-9:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18","timestamp":"2020-05-24 13:59:10.203959616Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_k86zaCt0pVAFBXphA-HHAKO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328750.291133_139.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:10.291133261Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"status":{"status":"write"}}},{"id":"1_1590328750.444965_140.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:10.444965711Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328764.224998_141.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:24.224998965Z","data":{"read_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328764.442285_142.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:24.442285080Z","data":{"read_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328765.812508_143.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:25.812508979Z","data":{"read_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328765.976327_144.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:25.976327786Z","data":{"read_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328767.020248_145.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:27.020248606Z","data":{"read_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328767.216616_146.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:27.216616731Z","data":{"read_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":2},"status":{"status":"complete"}}}] 2020-05-24T15:06:53.026 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:53.027 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:53.028 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 114a3cfc-32a6-4ae9-bab5-baac4f59a79e --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:56.070 INFO:teuthology.orchestra.run.smithi160.stdout:[{"id":"1_1590328567.776739_38.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:07.776739104Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ui15Zsog5OeOBfQLFnhWmAY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328568.276896_39.1","section":"user","name":"rgw-multisite-test-user","timestamp":"2020-05-24 13:56:08.276896545Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ui15Zsog5OeOBfQLFnhWmAY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328780.710920_36.1","section":"bucket.instance","name":"ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","timestamp":"2020-05-24 13:59:40.710920367Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_1x1ZZjoXeMNzJoUjkg5bWmM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328780.907463_37.1","section":"bucket.instance","name":"ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","timestamp":"2020-05-24 13:59:40.907463503Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_1x1ZZjoXeMNzJoUjkg5bWmM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328780.992516_38.1","section":"bucket","name":"ajxnhd-10","timestamp":"2020-05-24 13:59:40.992516918Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_F0eV8p4gy4VVoOyBszXK9Uy","ver":1},"status":{"status":"write"}}},{"id":"1_1590328781.173823_39.1","section":"bucket","name":"ajxnhd-10","timestamp":"2020-05-24 13:59:41.173823390Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_F0eV8p4gy4VVoOyBszXK9Uy","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328781.786763_40.1","section":"bucket.instance","name":"ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","timestamp":"2020-05-24 13:59:41.786763813Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yAfrDCGApzmfUfpLV32eRYD","ver":1},"status":{"status":"write"}}},{"id":"1_1590328782.049055_41.1","section":"bucket.instance","name":"ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","timestamp":"2020-05-24 13:59:42.049055331Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yAfrDCGApzmfUfpLV32eRYD","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328782.155023_42.1","section":"bucket","name":"ajxnhd-11","timestamp":"2020-05-24 13:59:42.155023264Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6JZ7TwZSk3ER0WTRkMdpYxK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328782.443391_43.1","section":"bucket","name":"ajxnhd-11","timestamp":"2020-05-24 13:59:42.443391351Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6JZ7TwZSk3ER0WTRkMdpYxK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.411331_44.1","section":"bucket.instance","name":"ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","timestamp":"2020-05-24 13:59:43.411331039Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5HqOnT066cNskmDRsRqZ-dE","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.597981_45.1","section":"bucket.instance","name":"ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","timestamp":"2020-05-24 13:59:43.597981263Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_5HqOnT066cNskmDRsRqZ-dE","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328783.683466_46.1","section":"bucket","name":"ajxnhd-12","timestamp":"2020-05-24 13:59:43.683466272Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_z0ez1Ljq8OH081qJv8Q7CL6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328783.837755_47.1","section":"bucket","name":"ajxnhd-12","timestamp":"2020-05-24 13:59:43.837755756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_z0ez1Ljq8OH081qJv8Q7CL6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329320.125728_54.1","section":"bucket.instance","name":"ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","timestamp":"2020-05-24 14:08:40.125728151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NEqYrxBauZKMtGHHXFkIZCq","ver":1},"status":{"status":"write"}}},{"id":"1_1590329320.290517_55.1","section":"bucket.instance","name":"ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","timestamp":"2020-05-24 14:08:40.290517017Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NEqYrxBauZKMtGHHXFkIZCq","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329320.400997_56.1","section":"bucket","name":"ajxnhd-13","timestamp":"2020-05-24 14:08:40.400997847Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eweXCos3PdlUyYekfNFjSHH","ver":1},"status":{"status":"write"}}},{"id":"1_1590329320.572773_57.1","section":"bucket","name":"ajxnhd-13","timestamp":"2020-05-24 14:08:40.572773700Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eweXCos3PdlUyYekfNFjSHH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329322.157703_58.1","section":"bucket.instance","name":"ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","timestamp":"2020-05-24 14:08:42.157703103Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BmyQDJAOboCFiJ45lLtofKE","ver":1},"status":{"status":"write"}}},{"id":"1_1590329322.303444_59.1","section":"bucket.instance","name":"ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","timestamp":"2020-05-24 14:08:42.303444730Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BmyQDJAOboCFiJ45lLtofKE","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329322.485950_60.1","section":"bucket","name":"ajxnhd-14","timestamp":"2020-05-24 14:08:42.485950917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_quzz_j7V4et7RpGVfsZ8DI3","ver":1},"status":{"status":"write"}}},{"id":"1_1590329322.753097_61.1","section":"bucket","name":"ajxnhd-14","timestamp":"2020-05-24 14:08:42.753097194Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_quzz_j7V4et7RpGVfsZ8DI3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329323.912546_62.1","section":"bucket.instance","name":"ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","timestamp":"2020-05-24 14:08:43.912546421Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ljytbx-c-ioF_jOyI19KhIn","ver":1},"status":{"status":"write"}}},{"id":"1_1590329324.055027_63.1","section":"bucket.instance","name":"ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","timestamp":"2020-05-24 14:08:44.055027596Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ljytbx-c-ioF_jOyI19KhIn","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329324.140936_64.1","section":"bucket","name":"ajxnhd-15","timestamp":"2020-05-24 14:08:44.140936259Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__NTNWGbez_pXSCfSDkXBqXJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590329324.279876_65.1","section":"bucket","name":"ajxnhd-15","timestamp":"2020-05-24 14:08:44.279876745Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__NTNWGbez_pXSCfSDkXBqXJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329851.002187_72.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:31.002187986Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"status":{"status":"write"}}},{"id":"1_1590329851.192404_73.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:31.192404499Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329851.255970_74.1","section":"bucket","name":"ajxnhd-16","timestamp":"2020-05-24 14:17:31.255970913Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_n5pa9q08aZnglX0ZBX2MI72","ver":1},"status":{"status":"write"}}},{"id":"1_1590329851.450578_75.1","section":"bucket","name":"ajxnhd-16","timestamp":"2020-05-24 14:17:31.450578151Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_n5pa9q08aZnglX0ZBX2MI72","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329852.156682_76.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:32.156682053Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"status":{"status":"write"}}},{"id":"1_1590329852.314606_77.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:32.314606785Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329852.385946_78.1","section":"bucket","name":"ajxnhd-17","timestamp":"2020-05-24 14:17:32.385946764Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2yVMkwTR0dwxoaxsJDfspmT","ver":1},"status":{"status":"write"}}},{"id":"1_1590329852.537860_79.1","section":"bucket","name":"ajxnhd-17","timestamp":"2020-05-24 14:17:32.537860267Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_2yVMkwTR0dwxoaxsJDfspmT","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329853.921024_80.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:33.921024364Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"status":{"status":"write"}}},{"id":"1_1590329854.087205_81.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:34.087205756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329854.204871_82.1","section":"bucket","name":"ajxnhd-18","timestamp":"2020-05-24 14:17:34.204871979Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBs2gR836NQmjWKNfHbjG8K","ver":1},"status":{"status":"write"}}},{"id":"1_1590329854.445256_83.1","section":"bucket","name":"ajxnhd-18","timestamp":"2020-05-24 14:17:34.445256737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NBs2gR836NQmjWKNfHbjG8K","ver":1},"status":{"status":"complete"}}},{"id":"1_1590329855.079839_84.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:35.079839696Z","data":{"read_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":2},"status":{"status":"write"}}},{"id":"1_1590329855.330800_85.1","section":"bucket.instance","name":"ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","timestamp":"2020-05-24 14:17:35.330800723Z","data":{"read_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":1},"write_version":{"tag":"_VggBY_ZIcJ-dzLYXjm4odC9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329856.118614_86.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:36.118614343Z","data":{"read_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":2},"status":{"status":"write"}}},{"id":"1_1590329856.475914_87.1","section":"bucket.instance","name":"ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","timestamp":"2020-05-24 14:17:36.475914391Z","data":{"read_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":1},"write_version":{"tag":"_JQ3k4MwuaM4XrTBBRQ6c4HU","ver":2},"status":{"status":"complete"}}},{"id":"1_1590329857.623824_88.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:37.623824536Z","data":{"read_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":2},"status":{"status":"write"}}},{"id":"1_1590329857.827328_89.1","section":"bucket.instance","name":"ajxnhd-18:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27","timestamp":"2020-05-24 14:17:37.827328629Z","data":{"read_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":1},"write_version":{"tag":"__qSoGuzAB_X0EtwSL6hpCFx","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330379.734826_96.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:19.734826079Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"status":{"status":"write"}}},{"id":"1_1590330379.889098_97.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:19.889098852Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330380.055101_98.1","section":"bucket","name":"ajxnhd-19","timestamp":"2020-05-24 14:26:20.055101601Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VogH8d_v03-pKxveKvcQRpR","ver":1},"status":{"status":"write"}}},{"id":"1_1590330380.186271_99.1","section":"bucket","name":"ajxnhd-19","timestamp":"2020-05-24 14:26:20.186271981Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VogH8d_v03-pKxveKvcQRpR","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330380.342253_100.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:20.342253953Z","data":{"read_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":2},"status":{"status":"write"}}},{"id":"1_1590330380.583029_101.1","section":"bucket.instance","name":"ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","timestamp":"2020-05-24 14:26:20.583029777Z","data":{"read_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":1},"write_version":{"tag":"_DmzAKYNG01mMi4T7OAbXDDW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590330903.436321_102.1","section":"bucket.instance","name":"ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","timestamp":"2020-05-24 14:35:03.436321562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gAFo5R_jIeCyFq2LSsNo17_","ver":1},"status":{"status":"write"}}},{"id":"1_1590330903.717682_103.1","section":"bucket.instance","name":"ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","timestamp":"2020-05-24 14:35:03.717682381Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gAFo5R_jIeCyFq2LSsNo17_","ver":1},"status":{"status":"complete"}}},{"id":"1_1590330903.810200_104.1","section":"bucket","name":"ajxnhd-20","timestamp":"2020-05-24 14:35:03.810200796Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zOYKFnexj8DFjS8jtUKG4DJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590330903.938053_105.1","section":"bucket","name":"ajxnhd-20","timestamp":"2020-05-24 14:35:03.938053252Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zOYKFnexj8DFjS8jtUKG4DJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331429.252015_120.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:49.252015956Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"status":{"status":"write"}}},{"id":"1_1590331429.406239_121.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:49.406239502Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331429.478659_122.1","section":"bucket","name":"ajxnhd-21","timestamp":"2020-05-24 14:43:49.478659851Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LhHMDRFw1MbeqVRBXiLVGqV","ver":1},"status":{"status":"write"}}},{"id":"1_1590331429.775227_123.1","section":"bucket","name":"ajxnhd-21","timestamp":"2020-05-24 14:43:49.775227278Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LhHMDRFw1MbeqVRBXiLVGqV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331430.455137_124.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:50.455137998Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"status":{"status":"write"}}},{"id":"1_1590331430.680723_125.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:50.680723110Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331430.776858_126.1","section":"bucket","name":"ajxnhd-22","timestamp":"2020-05-24 14:43:50.776858522Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zS0Qv2MgKEFW14qiK5JnkWJ","ver":1},"status":{"status":"write"}}},{"id":"1_1590331430.897169_127.1","section":"bucket","name":"ajxnhd-22","timestamp":"2020-05-24 14:43:50.897169843Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zS0Qv2MgKEFW14qiK5JnkWJ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331432.003953_128.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:52.003953530Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"status":{"status":"write"}}},{"id":"1_1590331432.142214_129.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:52.142214510Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331432.205496_130.1","section":"bucket","name":"ajxnhd-23","timestamp":"2020-05-24 14:43:52.205496460Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yrznneKnovuxIXNnXnO0uLI","ver":1},"status":{"status":"write"}}},{"id":"1_1590331432.337983_131.1","section":"bucket","name":"ajxnhd-23","timestamp":"2020-05-24 14:43:52.337983599Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_yrznneKnovuxIXNnXnO0uLI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331433.714200_132.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:53.714200278Z","data":{"read_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":2},"status":{"status":"write"}}},{"id":"1_1590331433.948922_133.1","section":"bucket.instance","name":"ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","timestamp":"2020-05-24 14:43:53.948922752Z","data":{"read_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":1},"write_version":{"tag":"_BGtziCeCQHsy7_-t65n-ZTf","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331434.894781_134.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:54.894781681Z","data":{"read_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":2},"status":{"status":"write"}}},{"id":"1_1590331435.198978_135.1","section":"bucket.instance","name":"ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","timestamp":"2020-05-24 14:43:55.198978175Z","data":{"read_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":1},"write_version":{"tag":"_ceNTmT9U7ib9Ip1jlj6pkq-","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331435.928156_136.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:55.928156482Z","data":{"read_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":2},"status":{"status":"write"}}},{"id":"1_1590331436.244684_137.1","section":"bucket.instance","name":"ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","timestamp":"2020-05-24 14:43:56.244684179Z","data":{"read_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":1},"write_version":{"tag":"_eoBAK2GsCmJJFnZ93DnjOtw","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331983.049122_144.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:03.049122099Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"status":{"status":"write"}}},{"id":"1_1590331983.197037_145.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:03.197037858Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331983.266790_146.1","section":"bucket","name":"ajxnhd-24","timestamp":"2020-05-24 14:53:03.266790936Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uIlRzMpJio6QSnre8AMGFLm","ver":1},"status":{"status":"write"}}},{"id":"1_1590331983.424889_147.1","section":"bucket","name":"ajxnhd-24","timestamp":"2020-05-24 14:53:03.424889595Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uIlRzMpJio6QSnre8AMGFLm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331983.865257_148.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:03.865257449Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"status":{"status":"write"}}},{"id":"1_1590331984.122968_149.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:04.122968508Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331984.251929_150.1","section":"bucket","name":"ajxnhd-25","timestamp":"2020-05-24 14:53:04.251929448Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uQJr9MjpeQurPAVQIXSW_JM","ver":1},"status":{"status":"write"}}},{"id":"1_1590331984.505736_151.1","section":"bucket","name":"ajxnhd-25","timestamp":"2020-05-24 14:53:04.505736130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_uQJr9MjpeQurPAVQIXSW_JM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331986.438990_152.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:06.438990629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"status":{"status":"write"}}},{"id":"1_1590331986.585637_153.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:06.585637806Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331986.647556_154.1","section":"bucket","name":"ajxnhd-26","timestamp":"2020-05-24 14:53:06.647556290Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Al_0YQq_jeAkOhBQMDkf2Z7","ver":1},"status":{"status":"write"}}},{"id":"1_1590331986.814186_155.1","section":"bucket","name":"ajxnhd-26","timestamp":"2020-05-24 14:53:06.814186094Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Al_0YQq_jeAkOhBQMDkf2Z7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590331988.439385_156.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.439385111Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"status":{"status":"write"}}},{"id":"1_1590331988.794207_157.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.794207447Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":1},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331988.879023_158.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:08.879023435Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":3},"status":{"status":"write"}}},{"id":"1_1590331989.060960_159.1","section":"bucket.instance","name":"ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","timestamp":"2020-05-24 14:53:09.060960790Z","data":{"read_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":2},"write_version":{"tag":"_wrSmbUAQAsySW8fHicGBNXw","ver":3},"status":{"status":"complete"}}},{"id":"1_1590331989.527880_160.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:09.527880038Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"status":{"status":"write"}}},{"id":"1_1590331989.813285_161.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:09.813285672Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":1},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331990.123113_162.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:10.123113736Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":3},"status":{"status":"write"}}},{"id":"1_1590331990.429901_163.1","section":"bucket.instance","name":"ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","timestamp":"2020-05-24 14:53:10.429901443Z","data":{"read_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":2},"write_version":{"tag":"_u363JoOMxe3wvyApB3UJWQN","ver":3},"status":{"status":"complete"}}},{"id":"1_1590331991.738750_164.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:11.738750891Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"status":{"status":"write"}}},{"id":"1_1590331991.926433_165.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:11.926433769Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":1},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"status":{"status":"complete"}}},{"id":"1_1590331992.233007_166.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:12.233007240Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":3},"status":{"status":"write"}}},{"id":"1_1590331992.462010_167.1","section":"bucket.instance","name":"ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","timestamp":"2020-05-24 14:53:12.462010498Z","data":{"read_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":2},"write_version":{"tag":"_RZrm99gHDde4fQMiaDa-40Z","ver":3},"status":{"status":"complete"}}},{"id":"1_1590332542.631802_177.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:22.631802311Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"status":{"status":"write"}}},{"id":"1_1590332542.830206_178.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:22.830206865Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332542.936016_179.1","section":"bucket","name":"ajxnhd-27","timestamp":"2020-05-24 15:02:22.936016582Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6-ZQ1R4_Nnl5GdcjPwHQgoI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332543.103013_180.1","section":"bucket","name":"ajxnhd-27","timestamp":"2020-05-24 15:02:23.103013361Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_6-ZQ1R4_Nnl5GdcjPwHQgoI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332543.775583_181.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:23.775583592Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"status":{"status":"write"}}},{"id":"1_1590332544.023476_182.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:24.023476172Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332544.120691_183.1","section":"bucket","name":"ajxnhd-28","timestamp":"2020-05-24 15:02:24.120691142Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_S_0X21LYEXlkSLS9-UznAcI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332544.302282_184.1","section":"bucket","name":"ajxnhd-28","timestamp":"2020-05-24 15:02:24.302282945Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_S_0X21LYEXlkSLS9-UznAcI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332545.404781_185.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:25.404781963Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332545.585733_186.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:25.585733302Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332545.654256_187.1","section":"bucket","name":"ajxnhd-29","timestamp":"2020-05-24 15:02:25.654256391Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vqZzPK8m08iLaKomZNlavH0","ver":1},"status":{"status":"write"}}},{"id":"1_1590332545.771078_188.1","section":"bucket","name":"ajxnhd-29","timestamp":"2020-05-24 15:02:25.771078802Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vqZzPK8m08iLaKomZNlavH0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332546.555221_189.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:26.555221665Z","data":{"read_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":2},"status":{"status":"write"}}},{"id":"1_1590332546.717101_190.1","section":"bucket.instance","name":"ajxnhd-27:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36","timestamp":"2020-05-24 15:02:26.717101024Z","data":{"read_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":1},"write_version":{"tag":"_4wTv16Vup3Ms9z_5SqxzEqk","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332547.859104_191.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:27.859104280Z","data":{"read_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":2},"status":{"status":"write"}}},{"id":"1_1590332548.198444_192.1","section":"bucket.instance","name":"ajxnhd-28:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37","timestamp":"2020-05-24 15:02:28.198444891Z","data":{"read_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":1},"write_version":{"tag":"_mIVy9L6QoyMjfeqMdFPpxMM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332548.675933_193.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:28.675933514Z","data":{"read_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":2},"status":{"status":"write"}}},{"id":"1_1590332549.028549_194.1","section":"bucket.instance","name":"ajxnhd-29:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38","timestamp":"2020-05-24 15:02:29.028549384Z","data":{"read_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":1},"write_version":{"tag":"_9D7ydBCAzx5WLbJYZD0O-uG","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332549.967216_195.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:29.967216514Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"status":{"status":"write"}}},{"id":"1_1590332550.115459_196.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:30.115459573Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332550.244077_197.1","section":"bucket","name":"ajxnhd-30","timestamp":"2020-05-24 15:02:30.244077484Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P61wwDe_JM5YuqmV6LehKBK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332550.456855_198.1","section":"bucket","name":"ajxnhd-30","timestamp":"2020-05-24 15:02:30.456855328Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_P61wwDe_JM5YuqmV6LehKBK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332551.822124_199.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:31.822124501Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"status":{"status":"write"}}},{"id":"1_1590332552.065161_200.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:32.065161497Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332552.144138_201.1","section":"bucket","name":"ajxnhd-31","timestamp":"2020-05-24 15:02:32.144138917Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_guBR1c3-SAHv8LKMh2QLb7A","ver":1},"status":{"status":"write"}}},{"id":"1_1590332552.260354_202.1","section":"bucket","name":"ajxnhd-31","timestamp":"2020-05-24 15:02:32.260354003Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_guBR1c3-SAHv8LKMh2QLb7A","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332553.270432_203.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:33.270432895Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"status":{"status":"write"}}},{"id":"1_1590332553.415925_204.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:33.415925128Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332553.486668_205.1","section":"bucket","name":"ajxnhd-32","timestamp":"2020-05-24 15:02:33.486668591Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_s1Byxm5NDVFw_jUWDF_FjgS","ver":1},"status":{"status":"write"}}},{"id":"1_1590332553.660263_206.1","section":"bucket","name":"ajxnhd-32","timestamp":"2020-05-24 15:02:33.660263701Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_s1Byxm5NDVFw_jUWDF_FjgS","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332554.995884_207.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:34.995884736Z","data":{"read_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":2},"status":{"status":"write"}}},{"id":"1_1590332555.358899_208.1","section":"bucket.instance","name":"ajxnhd-30:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39","timestamp":"2020-05-24 15:02:35.358899184Z","data":{"read_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":1},"write_version":{"tag":"_nsy-sF7HWIj_yKAwK6jydNj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332556.426757_209.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:36.426757678Z","data":{"read_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":2},"status":{"status":"write"}}},{"id":"1_1590332556.687130_210.1","section":"bucket.instance","name":"ajxnhd-31:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40","timestamp":"2020-05-24 15:02:36.687130128Z","data":{"read_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":1},"write_version":{"tag":"_0EsfbGtA7hRpWUXofiv4Kgm","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332557.695414_211.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:37.695414905Z","data":{"read_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":2},"status":{"status":"write"}}},{"id":"1_1590332557.981422_212.1","section":"bucket.instance","name":"ajxnhd-32:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41","timestamp":"2020-05-24 15:02:37.981422719Z","data":{"read_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":1},"write_version":{"tag":"_FjbfvurDz4B_WUeLdlFd_Y9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332559.305198_213.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:39.305198079Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"status":{"status":"write"}}},{"id":"1_1590332559.482651_214.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:39.482651412Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332559.540073_215.1","section":"bucket","name":"ajxnhd-33","timestamp":"2020-05-24 15:02:39.540073474Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__TXIL7F0fo6f_ML5KkyR5Cf","ver":1},"status":{"status":"write"}}},{"id":"1_1590332559.677626_216.1","section":"bucket","name":"ajxnhd-33","timestamp":"2020-05-24 15:02:39.677626433Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"__TXIL7F0fo6f_ML5KkyR5Cf","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332560.881764_217.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:40.881764475Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"status":{"status":"write"}}},{"id":"1_1590332561.064185_218.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:41.064185535Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332561.144892_219.1","section":"bucket","name":"ajxnhd-34","timestamp":"2020-05-24 15:02:41.144892141Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TnCet-EC2ujq4RjCMmtoi5b","ver":1},"status":{"status":"write"}}},{"id":"1_1590332561.397068_220.1","section":"bucket","name":"ajxnhd-34","timestamp":"2020-05-24 15:02:41.397068589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_TnCet-EC2ujq4RjCMmtoi5b","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332562.761328_221.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:42.761328654Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"status":{"status":"write"}}},{"id":"1_1590332562.887604_222.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:42.887604521Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332562.947140_223.1","section":"bucket","name":"ajxnhd-35","timestamp":"2020-05-24 15:02:42.947140222Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4J9adjb6Qa0J6bXq6rh1nYz","ver":1},"status":{"status":"write"}}},{"id":"1_1590332563.062797_224.1","section":"bucket","name":"ajxnhd-35","timestamp":"2020-05-24 15:02:43.062797471Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4J9adjb6Qa0J6bXq6rh1nYz","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332564.442927_225.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:44.442927388Z","data":{"read_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":2},"status":{"status":"write"}}},{"id":"1_1590332564.792134_226.1","section":"bucket.instance","name":"ajxnhd-33:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42","timestamp":"2020-05-24 15:02:44.792134507Z","data":{"read_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":1},"write_version":{"tag":"_gNwvR2JER0qqcUbn6HOIaik","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332566.229361_227.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:46.229361522Z","data":{"read_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":2},"status":{"status":"write"}}},{"id":"1_1590332566.526084_228.1","section":"bucket.instance","name":"ajxnhd-34:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43","timestamp":"2020-05-24 15:02:46.526084519Z","data":{"read_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":1},"write_version":{"tag":"_VFO_z7lhduCEWoZaYnyuXg9","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332568.689676_229.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:48.689676599Z","data":{"read_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":2},"status":{"status":"write"}}},{"id":"1_1590332568.878454_230.1","section":"bucket.instance","name":"ajxnhd-35:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44","timestamp":"2020-05-24 15:02:48.878454727Z","data":{"read_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":1},"write_version":{"tag":"_Mp1iQUnY9BzE9vMRQ2mfZez","ver":2},"status":{"status":"complete"}}},{"id":"1_1590332570.049297_231.1","section":"bucket.instance","name":"ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","timestamp":"2020-05-24 15:02:50.049297529Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_b9k2iMA1NtKXNRMZLjKnuMK","ver":1},"status":{"status":"write"}}},{"id":"1_1590332570.244663_232.1","section":"bucket.instance","name":"ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","timestamp":"2020-05-24 15:02:50.244663844Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_b9k2iMA1NtKXNRMZLjKnuMK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332570.372091_233.1","section":"bucket","name":"ajxnhd-36","timestamp":"2020-05-24 15:02:50.372091228Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7Jm22K8v7V5ENeX7fvvO6m3","ver":1},"status":{"status":"write"}}},{"id":"1_1590332570.524216_234.1","section":"bucket","name":"ajxnhd-36","timestamp":"2020-05-24 15:02:50.524216691Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_7Jm22K8v7V5ENeX7fvvO6m3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332570.961122_235.1","section":"bucket.instance","name":"ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","timestamp":"2020-05-24 15:02:50.961122188Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_24WmxNrnrGYWY6wlu6B-ieA","ver":1},"status":{"status":"write"}}},{"id":"1_1590332571.260025_236.1","section":"bucket.instance","name":"ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","timestamp":"2020-05-24 15:02:51.260025363Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_24WmxNrnrGYWY6wlu6B-ieA","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332571.358013_237.1","section":"bucket","name":"ajxnhd-37","timestamp":"2020-05-24 15:02:51.358013136Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sEb0Ve1wRSuEJNe56JIG4EI","ver":1},"status":{"status":"write"}}},{"id":"1_1590332571.530066_238.1","section":"bucket","name":"ajxnhd-37","timestamp":"2020-05-24 15:02:51.530066130Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sEb0Ve1wRSuEJNe56JIG4EI","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332573.119556_239.1","section":"bucket.instance","name":"ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","timestamp":"2020-05-24 15:02:53.119556405Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7hfIFt8sICEW9pyMqr-MYV","ver":1},"status":{"status":"write"}}},{"id":"1_1590332573.260989_240.1","section":"bucket.instance","name":"ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","timestamp":"2020-05-24 15:02:53.260989054Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Y7hfIFt8sICEW9pyMqr-MYV","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332573.335527_241.1","section":"bucket","name":"ajxnhd-38","timestamp":"2020-05-24 15:02:53.335527083Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vNQMrHjRLeJ3QZbyKgH0O_6","ver":1},"status":{"status":"write"}}},{"id":"1_1590332573.500710_242.1","section":"bucket","name":"ajxnhd-38","timestamp":"2020-05-24 15:02:53.500710562Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vNQMrHjRLeJ3QZbyKgH0O_6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332591.965748_243.1","section":"bucket.instance","name":"ajxnhd-39:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48","timestamp":"2020-05-24 15:03:11.965748944Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y58mPDPi7qXqG_CmkdmR6os","ver":1},"status":{"status":"write"}}},{"id":"1_1590332592.803259_244.1","section":"bucket.instance","name":"ajxnhd-39:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48","timestamp":"2020-05-24 15:03:12.803259188Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_y58mPDPi7qXqG_CmkdmR6os","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332593.312415_245.1","section":"bucket","name":"ajxnhd-39","timestamp":"2020-05-24 15:03:13.312415629Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UNwrQUdBwy_B1nsJ18BGrXu","ver":1},"status":{"status":"write"}}},{"id":"1_1590332593.915640_246.1","section":"bucket","name":"ajxnhd-39","timestamp":"2020-05-24 15:03:13.915640047Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UNwrQUdBwy_B1nsJ18BGrXu","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332595.681045_247.1","section":"bucket.instance","name":"ajxnhd-40:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49","timestamp":"2020-05-24 15:03:15.681045977Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cGbybOYRxqaUODITnQ64wX8","ver":1},"status":{"status":"write"}}},{"id":"1_1590332595.965278_248.1","section":"bucket.instance","name":"ajxnhd-40:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49","timestamp":"2020-05-24 15:03:15.965278368Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cGbybOYRxqaUODITnQ64wX8","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332596.121270_249.1","section":"bucket","name":"ajxnhd-40","timestamp":"2020-05-24 15:03:16.121270922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WSY76UVteF1dCj_5H4jHX5D","ver":1},"status":{"status":"write"}}},{"id":"1_1590332596.288693_250.1","section":"bucket","name":"ajxnhd-40","timestamp":"2020-05-24 15:03:16.288693632Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_WSY76UVteF1dCj_5H4jHX5D","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332597.688431_251.1","section":"bucket.instance","name":"ajxnhd-41:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50","timestamp":"2020-05-24 15:03:17.688431472Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AK-EzZL3ZkVBaaCNyKRG8cA","ver":1},"status":{"status":"write"}}},{"id":"1_1590332597.968361_252.1","section":"bucket.instance","name":"ajxnhd-41:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50","timestamp":"2020-05-24 15:03:17.968361011Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AK-EzZL3ZkVBaaCNyKRG8cA","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332598.088015_253.1","section":"bucket","name":"ajxnhd-41","timestamp":"2020-05-24 15:03:18.088015374Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gs-0jKiD8zj7WuELPIX45Fk","ver":1},"status":{"status":"write"}}},{"id":"1_1590332598.292976_254.1","section":"bucket","name":"ajxnhd-41","timestamp":"2020-05-24 15:03:18.292976652Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Gs-0jKiD8zj7WuELPIX45Fk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328661.857046_44.1","section":"bucket.instance","name":"ajxnhd-1:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1","timestamp":"2020-05-24 13:57:41.857046576Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EI_f_0Ki9favYOjyGg-58P0","ver":1},"status":{"status":"write"}}},{"id":"1_1590328663.801409_45.1","section":"bucket.instance","name":"ajxnhd-1:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1","timestamp":"2020-05-24 13:57:43.801409385Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_EI_f_0Ki9favYOjyGg-58P0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328665.095008_46.1","section":"bucket","name":"ajxnhd-1","timestamp":"2020-05-24 13:57:45.095008350Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZoWBta9CxnaXqf3Hnfk32TH","ver":1},"status":{"status":"write"}}},{"id":"1_1590328666.568066_47.1","section":"bucket","name":"ajxnhd-1","timestamp":"2020-05-24 13:57:46.568066655Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ZoWBta9CxnaXqf3Hnfk32TH","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328669.388156_48.1","section":"bucket.instance","name":"ajxnhd-2:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2","timestamp":"2020-05-24 13:57:49.388156020Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GLHHyqK2R6W2PTMOkPNrrlk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328671.962074_49.1","section":"bucket.instance","name":"ajxnhd-2:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2","timestamp":"2020-05-24 13:57:51.962074124Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_GLHHyqK2R6W2PTMOkPNrrlk","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328674.256018_50.1","section":"bucket","name":"ajxnhd-2","timestamp":"2020-05-24 13:57:54.256018718Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGN9Rv5tiGtmUXakupLtXPZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328676.357935_51.1","section":"bucket","name":"ajxnhd-2","timestamp":"2020-05-24 13:57:56.357935846Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_CGN9Rv5tiGtmUXakupLtXPZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328681.147077_52.1","section":"bucket.instance","name":"ajxnhd-3:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3","timestamp":"2020-05-24 13:58:01.147077733Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9TpTYOx5Q03ed_c9goFYJZM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328681.299764_53.1","section":"bucket.instance","name":"ajxnhd-3:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3","timestamp":"2020-05-24 13:58:01.299764826Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_9TpTYOx5Q03ed_c9goFYJZM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328681.371143_54.1","section":"bucket","name":"ajxnhd-3","timestamp":"2020-05-24 13:58:01.371143798Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vPul70r_8Ep8FRLwG3vWyNZ","ver":1},"status":{"status":"write"}}},{"id":"1_1590328681.510269_55.1","section":"bucket","name":"ajxnhd-3","timestamp":"2020-05-24 13:58:01.510269305Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vPul70r_8Ep8FRLwG3vWyNZ","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328695.908700_56.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4","timestamp":"2020-05-24 13:58:15.908700393Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vLXsYr9SmfTZ3Fb3d_wnAd3","ver":1},"status":{"status":"write"}}},{"id":"1_1590328696.114303_57.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4","timestamp":"2020-05-24 13:58:16.114303048Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vLXsYr9SmfTZ3Fb3d_wnAd3","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328696.194626_58.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:16.194626894Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Jg0IeWtO91dabgJ-_9OekxY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328696.347412_59.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:16.347412194Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Jg0IeWtO91dabgJ-_9OekxY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328697.502176_60.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5","timestamp":"2020-05-24 13:58:17.502176684Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LqXlxfl8QRw7-uDVpsgnDcw","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.668291_61.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5","timestamp":"2020-05-24 13:58:17.668291982Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LqXlxfl8QRw7-uDVpsgnDcw","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328697.761208_62.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:17.761208203Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_obyOfpY3MrTzY_e23jlE70I","ver":1},"status":{"status":"write"}}},{"id":"1_1590328697.902648_63.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:17.902648222Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_obyOfpY3MrTzY_e23jlE70I","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328699.527169_64.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6","timestamp":"2020-05-24 13:58:19.527169223Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_o4Ky02fbN6YWn_f_ggrN7C7","ver":1},"status":{"status":"write"}}},{"id":"1_1590328699.677825_65.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6","timestamp":"2020-05-24 13:58:19.677825821Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_o4Ky02fbN6YWn_f_ggrN7C7","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328699.833869_66.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:19.833869190Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_enfzxiViqv_w3BWd10aOZ0d","ver":1},"status":{"status":"write"}}},{"id":"1_1590328700.029897_67.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:20.029897018Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_enfzxiViqv_w3BWd10aOZ0d","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328714.515118_70.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.515118726Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"status":{"status":"write"}}},{"id":"1_1590328714.719389_71.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.719389695Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328714.782348_72.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:34.782348779Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4HVX6Tlmf_oEsDt4DBAWClI","ver":1},"status":{"status":"write"}}},{"id":"1_1590328714.897108_73.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:34.897108137Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_4HVX6Tlmf_oEsDt4DBAWClI","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328714.993989_74.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:34.993989083Z","data":{"read_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328715.172258_75.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.7","timestamp":"2020-05-24 13:58:35.172258827Z","data":{"read_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":1},"write_version":{"tag":"_IcOkhDKC1I0wzAXS2JyQQrj","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328715.564378_76.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:35.564378963Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"status":{"status":"write"}}},{"id":"1_1590328715.749934_77.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:35.749934229Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328715.857497_78.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:35.857497048Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NJuC_NimR8vZNocPMx3wfsi","ver":1},"status":{"status":"write"}}},{"id":"1_1590328716.050040_79.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:36.050040815Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_NJuC_NimR8vZNocPMx3wfsi","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328716.187601_80.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:36.187601735Z","data":{"read_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328716.382658_81.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.8","timestamp":"2020-05-24 13:58:36.382658090Z","data":{"read_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":1},"write_version":{"tag":"_zjYpcPD7W3h3ttkTZcIJ9D2","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328716.732842_82.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:36.732842955Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"status":{"status":"write"}}},{"id":"1_1590328717.017969_83.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.017969634Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328717.178040_84.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:37.178040274Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sCm2MGtfgkAIqwwkQk3Yw5q","ver":1},"status":{"status":"write"}}},{"id":"1_1590328717.373287_85.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:37.373287652Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_sCm2MGtfgkAIqwwkQk3Yw5q","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328717.510560_86.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.510560445Z","data":{"read_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328717.684607_87.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.9","timestamp":"2020-05-24 13:58:37.684607007Z","data":{"read_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":1},"write_version":{"tag":"_Ynh472dlBj291DR7Z2TqLcY","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328718.317405_89.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.317405148Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"status":{"status":"write"}}},{"id":"1_1590328718.504832_90.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.504832890Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328718.571111_91.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:38.571111899Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e8rj40fpi3-JZbsFLxS45Xk","ver":1},"status":{"status":"write"}}},{"id":"1_1590328718.700562_92.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:38.700562070Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_e8rj40fpi3-JZbsFLxS45Xk","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328718.860296_93.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:38.860296964Z","data":{"read_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328719.136857_94.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.10","timestamp":"2020-05-24 13:58:39.136857916Z","data":{"read_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":1},"write_version":{"tag":"_cIvWcmkEw7pLjKkSyYwyd1u","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328720.315474_95.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.315474784Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"status":{"status":"write"}}},{"id":"1_1590328720.499875_96.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.499875922Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328720.583970_97.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:40.583970910Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xi3oOUty4bv3bHoncH0dIz3","ver":1},"status":{"status":"write"}}},{"id":"1_1590328720.753543_98.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:40.753543145Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_xi3oOUty4bv3bHoncH0dIz3","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328720.994211_99.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:40.994211183Z","data":{"read_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328721.267286_100.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.11","timestamp":"2020-05-24 13:58:41.267286430Z","data":{"read_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":1},"write_version":{"tag":"_32Uhl38SlNe4oYvWuRuPzkt","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328722.062273_101.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.062273899Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"status":{"status":"write"}}},{"id":"1_1590328722.334638_102.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.334638645Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328722.416735_103.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:42.416735183Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_dVkfYwfTCvS5gmH63FscSTy","ver":1},"status":{"status":"write"}}},{"id":"1_1590328722.542035_104.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:42.542035756Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_dVkfYwfTCvS5gmH63FscSTy","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328722.640315_105.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.640315136Z","data":{"read_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328722.943793_106.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.12","timestamp":"2020-05-24 13:58:42.943793752Z","data":{"read_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":1},"write_version":{"tag":"_LVXP5MsvQnTRxS-9r5RxVVK","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328724.614660_108.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:44.614660660Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328724.809740_109.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:44.809740816Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328724.882230_110.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:44.882230120Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UlCm7CE1xo5AsxtoMJbTMz4","ver":1},"status":{"status":"write"}}},{"id":"1_1590328725.087079_111.1","section":"bucket","name":"ajxnhd-4","timestamp":"2020-05-24 13:58:45.087079418Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_UlCm7CE1xo5AsxtoMJbTMz4","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328725.239107_112.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:45.239107841Z","data":{"read_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328725.441830_113.1","section":"bucket.instance","name":"ajxnhd-4:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.13","timestamp":"2020-05-24 13:58:45.441830031Z","data":{"read_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":1},"write_version":{"tag":"_eSPT77uDYHNH6eJwbAAZYUW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328726.292849_114.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.292849280Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"status":{"status":"write"}}},{"id":"1_1590328726.443715_115.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.443715821Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328726.524327_116.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:46.524327926Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VBLmoFhcIYIVqpzreE0XnKi","ver":1},"status":{"status":"write"}}},{"id":"1_1590328726.674271_117.1","section":"bucket","name":"ajxnhd-5","timestamp":"2020-05-24 13:58:46.674271737Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_VBLmoFhcIYIVqpzreE0XnKi","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328726.787812_118.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:46.787812842Z","data":{"read_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328727.010860_119.1","section":"bucket.instance","name":"ajxnhd-5:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.14","timestamp":"2020-05-24 13:58:47.010860196Z","data":{"read_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":1},"write_version":{"tag":"_knfdg-pgtrkYXmpSH5xZoop","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328727.922311_120.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:47.922311552Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"status":{"status":"write"}}},{"id":"1_1590328728.057327_121.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.057327847Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328728.160326_122.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:48.160326113Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u5I6Hkn6tF0zFwYrBt4lf2D","ver":1},"status":{"status":"write"}}},{"id":"1_1590328728.345721_123.1","section":"bucket","name":"ajxnhd-6","timestamp":"2020-05-24 13:58:48.345721203Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_u5I6Hkn6tF0zFwYrBt4lf2D","ver":1},"status":{"status":"abort"}}},{"id":"1_1590328728.436742_124.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.436742651Z","data":{"read_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328728.582516_125.1","section":"bucket.instance","name":"ajxnhd-6:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.15","timestamp":"2020-05-24 13:58:48.582516268Z","data":{"read_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":1},"write_version":{"tag":"_tDgs8_zAkCYUubuo-ttkDVM","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328744.824204_129.1","section":"bucket.instance","name":"ajxnhd-7:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16","timestamp":"2020-05-24 13:59:04.824204578Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_azYpBimd6nndafzvDBhV2e6","ver":1},"status":{"status":"write"}}},{"id":"1_1590328745.009557_130.1","section":"bucket.instance","name":"ajxnhd-7:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16","timestamp":"2020-05-24 13:59:05.009557060Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_azYpBimd6nndafzvDBhV2e6","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328745.089321_131.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:05.089321631Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"status":{"status":"write"}}},{"id":"1_1590328745.283838_132.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:05.283838511Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328746.557992_133.1","section":"bucket.instance","name":"ajxnhd-8:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17","timestamp":"2020-05-24 13:59:06.557992864Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iJutw-w353HxM0wQe0ahGW0","ver":1},"status":{"status":"write"}}},{"id":"1_1590328746.745740_134.1","section":"bucket.instance","name":"ajxnhd-8:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17","timestamp":"2020-05-24 13:59:06.745740588Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_iJutw-w353HxM0wQe0ahGW0","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328746.807024_135.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:06.807024179Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"status":{"status":"write"}}},{"id":"1_1590328747.035908_136.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:07.035908477Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328749.986118_137.1","section":"bucket.instance","name":"ajxnhd-9:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18","timestamp":"2020-05-24 13:59:09.986118602Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_k86zaCt0pVAFBXphA-HHAKO","ver":1},"status":{"status":"write"}}},{"id":"1_1590328750.203959_138.1","section":"bucket.instance","name":"ajxnhd-9:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18","timestamp":"2020-05-24 13:59:10.203959616Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_k86zaCt0pVAFBXphA-HHAKO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328750.291133_139.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:10.291133261Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"status":{"status":"write"}}},{"id":"1_1590328750.444965_140.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:10.444965711Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"status":{"status":"complete"}}},{"id":"1_1590328764.224998_141.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:24.224998965Z","data":{"read_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328764.442285_142.1","section":"bucket","name":"ajxnhd-7","timestamp":"2020-05-24 13:59:24.442285080Z","data":{"read_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":1},"write_version":{"tag":"_pkR2T2afZSn0F9cUQEZkdOW","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328765.812508_143.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:25.812508979Z","data":{"read_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328765.976327_144.1","section":"bucket","name":"ajxnhd-8","timestamp":"2020-05-24 13:59:25.976327786Z","data":{"read_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":1},"write_version":{"tag":"_t4goJBmnmA-kjeh1UbMfH1t","ver":2},"status":{"status":"complete"}}},{"id":"1_1590328767.020248_145.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:27.020248606Z","data":{"read_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":2},"status":{"status":"remove"}}},{"id":"1_1590328767.216616_146.1","section":"bucket","name":"ajxnhd-9","timestamp":"2020-05-24 13:59:27.216616731Z","data":{"read_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":1},"write_version":{"tag":"_vRjrQOVN1rwJdGgNcPfJYPx","ver":2},"status":{"status":"complete"}}}] 2020-05-24T15:06:56.074 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:56.074 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:56.075 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 7920f6a5-aa4e-4bd1-aca6-86793c8b287f --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:06:59.087 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":"1_1590332682.625373_153.1","section":"bucket.instance","name":"ajxnhd-42:8de99cfe-4678-427d-bca8-2f77415e7812.8073.1","timestamp":"2020-05-24 15:04:42.625373229Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DxIxF2G5UXFlTmctmjcLst5","ver":1},"status":{"status":"write"}}},{"id":"1_1590332682.784727_154.1","section":"bucket.instance","name":"ajxnhd-42:8de99cfe-4678-427d-bca8-2f77415e7812.8073.1","timestamp":"2020-05-24 15:04:42.784727589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DxIxF2G5UXFlTmctmjcLst5","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332682.851000_155.1","section":"bucket","name":"ajxnhd-42","timestamp":"2020-05-24 15:04:42.851000550Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_f1O132aCzwIBF0EQNSnDqNP","ver":1},"status":{"status":"write"}}},{"id":"1_1590332682.995438_156.1","section":"bucket","name":"ajxnhd-42","timestamp":"2020-05-24 15:04:42.995438014Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_f1O132aCzwIBF0EQNSnDqNP","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332683.849590_157.1","section":"bucket.instance","name":"ajxnhd-43:8de99cfe-4678-427d-bca8-2f77415e7812.8073.2","timestamp":"2020-05-24 15:04:43.849590908Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QEUe8pSxcRTDsWjTAVRaS0a","ver":1},"status":{"status":"write"}}},{"id":"1_1590332683.977137_158.1","section":"bucket.instance","name":"ajxnhd-43:8de99cfe-4678-427d-bca8-2f77415e7812.8073.2","timestamp":"2020-05-24 15:04:43.977137837Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QEUe8pSxcRTDsWjTAVRaS0a","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332684.033247_159.1","section":"bucket","name":"ajxnhd-43","timestamp":"2020-05-24 15:04:44.033247102Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AiGHZOKjX_CFU5RCo8Q_BvO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332684.147703_160.1","section":"bucket","name":"ajxnhd-43","timestamp":"2020-05-24 15:04:44.147703352Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AiGHZOKjX_CFU5RCo8Q_BvO","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:06:59.088 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:59.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:06:59.090 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 7920f6a5-aa4e-4bd1-aca6-86793c8b287f --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:01.389 INFO:teuthology.orchestra.run.smithi160.stdout:[{"id":"1_1590332682.625373_153.1","section":"bucket.instance","name":"ajxnhd-42:8de99cfe-4678-427d-bca8-2f77415e7812.8073.1","timestamp":"2020-05-24 15:04:42.625373229Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DxIxF2G5UXFlTmctmjcLst5","ver":1},"status":{"status":"write"}}},{"id":"1_1590332682.784727_154.1","section":"bucket.instance","name":"ajxnhd-42:8de99cfe-4678-427d-bca8-2f77415e7812.8073.1","timestamp":"2020-05-24 15:04:42.784727589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DxIxF2G5UXFlTmctmjcLst5","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332682.851000_155.1","section":"bucket","name":"ajxnhd-42","timestamp":"2020-05-24 15:04:42.851000550Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_f1O132aCzwIBF0EQNSnDqNP","ver":1},"status":{"status":"write"}}},{"id":"1_1590332682.995438_156.1","section":"bucket","name":"ajxnhd-42","timestamp":"2020-05-24 15:04:42.995438014Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_f1O132aCzwIBF0EQNSnDqNP","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332683.849590_157.1","section":"bucket.instance","name":"ajxnhd-43:8de99cfe-4678-427d-bca8-2f77415e7812.8073.2","timestamp":"2020-05-24 15:04:43.849590908Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QEUe8pSxcRTDsWjTAVRaS0a","ver":1},"status":{"status":"write"}}},{"id":"1_1590332683.977137_158.1","section":"bucket.instance","name":"ajxnhd-43:8de99cfe-4678-427d-bca8-2f77415e7812.8073.2","timestamp":"2020-05-24 15:04:43.977137837Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QEUe8pSxcRTDsWjTAVRaS0a","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332684.033247_159.1","section":"bucket","name":"ajxnhd-43","timestamp":"2020-05-24 15:04:44.033247102Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AiGHZOKjX_CFU5RCo8Q_BvO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332684.147703_160.1","section":"bucket","name":"ajxnhd-43","timestamp":"2020-05-24 15:04:44.147703352Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AiGHZOKjX_CFU5RCo8Q_BvO","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:01.390 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:01.391 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:01.392 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 7920f6a5-aa4e-4bd1-aca6-86793c8b287f --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:04.234 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":"1_1590332682.625373_153.1","section":"bucket.instance","name":"ajxnhd-42:8de99cfe-4678-427d-bca8-2f77415e7812.8073.1","timestamp":"2020-05-24 15:04:42.625373229Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DxIxF2G5UXFlTmctmjcLst5","ver":1},"status":{"status":"write"}}},{"id":"1_1590332682.784727_154.1","section":"bucket.instance","name":"ajxnhd-42:8de99cfe-4678-427d-bca8-2f77415e7812.8073.1","timestamp":"2020-05-24 15:04:42.784727589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DxIxF2G5UXFlTmctmjcLst5","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332682.851000_155.1","section":"bucket","name":"ajxnhd-42","timestamp":"2020-05-24 15:04:42.851000550Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_f1O132aCzwIBF0EQNSnDqNP","ver":1},"status":{"status":"write"}}},{"id":"1_1590332682.995438_156.1","section":"bucket","name":"ajxnhd-42","timestamp":"2020-05-24 15:04:42.995438014Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_f1O132aCzwIBF0EQNSnDqNP","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332683.849590_157.1","section":"bucket.instance","name":"ajxnhd-43:8de99cfe-4678-427d-bca8-2f77415e7812.8073.2","timestamp":"2020-05-24 15:04:43.849590908Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QEUe8pSxcRTDsWjTAVRaS0a","ver":1},"status":{"status":"write"}}},{"id":"1_1590332683.977137_158.1","section":"bucket.instance","name":"ajxnhd-43:8de99cfe-4678-427d-bca8-2f77415e7812.8073.2","timestamp":"2020-05-24 15:04:43.977137837Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QEUe8pSxcRTDsWjTAVRaS0a","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332684.033247_159.1","section":"bucket","name":"ajxnhd-43","timestamp":"2020-05-24 15:04:44.033247102Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AiGHZOKjX_CFU5RCo8Q_BvO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332684.147703_160.1","section":"bucket","name":"ajxnhd-43","timestamp":"2020-05-24 15:04:44.147703352Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AiGHZOKjX_CFU5RCo8Q_BvO","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:04.235 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:04.235 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:04.236 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 7920f6a5-aa4e-4bd1-aca6-86793c8b287f --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:06.893 INFO:teuthology.orchestra.run.smithi160.stdout:[{"id":"1_1590332682.625373_153.1","section":"bucket.instance","name":"ajxnhd-42:8de99cfe-4678-427d-bca8-2f77415e7812.8073.1","timestamp":"2020-05-24 15:04:42.625373229Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DxIxF2G5UXFlTmctmjcLst5","ver":1},"status":{"status":"write"}}},{"id":"1_1590332682.784727_154.1","section":"bucket.instance","name":"ajxnhd-42:8de99cfe-4678-427d-bca8-2f77415e7812.8073.1","timestamp":"2020-05-24 15:04:42.784727589Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_DxIxF2G5UXFlTmctmjcLst5","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332682.851000_155.1","section":"bucket","name":"ajxnhd-42","timestamp":"2020-05-24 15:04:42.851000550Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_f1O132aCzwIBF0EQNSnDqNP","ver":1},"status":{"status":"write"}}},{"id":"1_1590332682.995438_156.1","section":"bucket","name":"ajxnhd-42","timestamp":"2020-05-24 15:04:42.995438014Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_f1O132aCzwIBF0EQNSnDqNP","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332683.849590_157.1","section":"bucket.instance","name":"ajxnhd-43:8de99cfe-4678-427d-bca8-2f77415e7812.8073.2","timestamp":"2020-05-24 15:04:43.849590908Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QEUe8pSxcRTDsWjTAVRaS0a","ver":1},"status":{"status":"write"}}},{"id":"1_1590332683.977137_158.1","section":"bucket.instance","name":"ajxnhd-43:8de99cfe-4678-427d-bca8-2f77415e7812.8073.2","timestamp":"2020-05-24 15:04:43.977137837Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_QEUe8pSxcRTDsWjTAVRaS0a","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332684.033247_159.1","section":"bucket","name":"ajxnhd-43","timestamp":"2020-05-24 15:04:44.033247102Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AiGHZOKjX_CFU5RCo8Q_BvO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332684.147703_160.1","section":"bucket","name":"ajxnhd-43","timestamp":"2020-05-24 15:04:44.147703352Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_AiGHZOKjX_CFU5RCo8Q_BvO","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:06.894 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:06.894 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:06.895 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period f35208d4-7215-4668-86dd-a246cde34636 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:09.545 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":"1_1590332755.979602_102.1","section":"bucket.instance","name":"ajxnhd-44:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1","timestamp":"2020-05-24 15:05:55.979602317Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ItSrBIL5x2gd1yuDMCfo5EG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332756.104714_103.1","section":"bucket.instance","name":"ajxnhd-44:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1","timestamp":"2020-05-24 15:05:56.104714303Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ItSrBIL5x2gd1yuDMCfo5EG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332756.160613_104.1","section":"bucket","name":"ajxnhd-44","timestamp":"2020-05-24 15:05:56.160613150Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HjB0J3-_3Ecbwb6E9e1-pSe","ver":1},"status":{"status":"write"}}},{"id":"1_1590332756.290427_105.1","section":"bucket","name":"ajxnhd-44","timestamp":"2020-05-24 15:05:56.290427832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HjB0J3-_3Ecbwb6E9e1-pSe","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332757.010124_106.1","section":"bucket.instance","name":"ajxnhd-45:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2","timestamp":"2020-05-24 15:05:57.010124087Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_msmvDcyWLYbRK7uS3W_TAqO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332757.154995_107.1","section":"bucket.instance","name":"ajxnhd-45:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2","timestamp":"2020-05-24 15:05:57.154995444Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_msmvDcyWLYbRK7uS3W_TAqO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332757.220737_108.1","section":"bucket","name":"ajxnhd-45","timestamp":"2020-05-24 15:05:57.220737344Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-Be1tdZHgEt-kqvJvs319g7","ver":1},"status":{"status":"write"}}},{"id":"1_1590332757.336629_109.1","section":"bucket","name":"ajxnhd-45","timestamp":"2020-05-24 15:05:57.336629140Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-Be1tdZHgEt-kqvJvs319g7","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:09.546 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:09.547 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:09.547 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period f35208d4-7215-4668-86dd-a246cde34636 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:12.316 INFO:teuthology.orchestra.run.smithi160.stdout:[{"id":"1_1590332755.979602_102.1","section":"bucket.instance","name":"ajxnhd-44:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1","timestamp":"2020-05-24 15:05:55.979602317Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ItSrBIL5x2gd1yuDMCfo5EG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332756.104714_103.1","section":"bucket.instance","name":"ajxnhd-44:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1","timestamp":"2020-05-24 15:05:56.104714303Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ItSrBIL5x2gd1yuDMCfo5EG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332756.160613_104.1","section":"bucket","name":"ajxnhd-44","timestamp":"2020-05-24 15:05:56.160613150Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HjB0J3-_3Ecbwb6E9e1-pSe","ver":1},"status":{"status":"write"}}},{"id":"1_1590332756.290427_105.1","section":"bucket","name":"ajxnhd-44","timestamp":"2020-05-24 15:05:56.290427832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HjB0J3-_3Ecbwb6E9e1-pSe","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332757.010124_106.1","section":"bucket.instance","name":"ajxnhd-45:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2","timestamp":"2020-05-24 15:05:57.010124087Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_msmvDcyWLYbRK7uS3W_TAqO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332757.154995_107.1","section":"bucket.instance","name":"ajxnhd-45:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2","timestamp":"2020-05-24 15:05:57.154995444Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_msmvDcyWLYbRK7uS3W_TAqO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332757.220737_108.1","section":"bucket","name":"ajxnhd-45","timestamp":"2020-05-24 15:05:57.220737344Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-Be1tdZHgEt-kqvJvs319g7","ver":1},"status":{"status":"write"}}},{"id":"1_1590332757.336629_109.1","section":"bucket","name":"ajxnhd-45","timestamp":"2020-05-24 15:05:57.336629140Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-Be1tdZHgEt-kqvJvs319g7","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:12.317 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:12.318 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:12.319 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period f35208d4-7215-4668-86dd-a246cde34636 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:15.296 INFO:teuthology.orchestra.run.smithi050.stdout:[{"id":"1_1590332755.979602_102.1","section":"bucket.instance","name":"ajxnhd-44:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1","timestamp":"2020-05-24 15:05:55.979602317Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ItSrBIL5x2gd1yuDMCfo5EG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332756.104714_103.1","section":"bucket.instance","name":"ajxnhd-44:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1","timestamp":"2020-05-24 15:05:56.104714303Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ItSrBIL5x2gd1yuDMCfo5EG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332756.160613_104.1","section":"bucket","name":"ajxnhd-44","timestamp":"2020-05-24 15:05:56.160613150Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HjB0J3-_3Ecbwb6E9e1-pSe","ver":1},"status":{"status":"write"}}},{"id":"1_1590332756.290427_105.1","section":"bucket","name":"ajxnhd-44","timestamp":"2020-05-24 15:05:56.290427832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HjB0J3-_3Ecbwb6E9e1-pSe","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332757.010124_106.1","section":"bucket.instance","name":"ajxnhd-45:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2","timestamp":"2020-05-24 15:05:57.010124087Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_msmvDcyWLYbRK7uS3W_TAqO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332757.154995_107.1","section":"bucket.instance","name":"ajxnhd-45:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2","timestamp":"2020-05-24 15:05:57.154995444Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_msmvDcyWLYbRK7uS3W_TAqO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332757.220737_108.1","section":"bucket","name":"ajxnhd-45","timestamp":"2020-05-24 15:05:57.220737344Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-Be1tdZHgEt-kqvJvs319g7","ver":1},"status":{"status":"write"}}},{"id":"1_1590332757.336629_109.1","section":"bucket","name":"ajxnhd-45","timestamp":"2020-05-24 15:05:57.336629140Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-Be1tdZHgEt-kqvJvs319g7","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:15.297 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:15.297 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:15.298 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period f35208d4-7215-4668-86dd-a246cde34636 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:18.167 INFO:teuthology.orchestra.run.smithi160.stdout:[{"id":"1_1590332755.979602_102.1","section":"bucket.instance","name":"ajxnhd-44:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1","timestamp":"2020-05-24 15:05:55.979602317Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ItSrBIL5x2gd1yuDMCfo5EG","ver":1},"status":{"status":"write"}}},{"id":"1_1590332756.104714_103.1","section":"bucket.instance","name":"ajxnhd-44:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1","timestamp":"2020-05-24 15:05:56.104714303Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_ItSrBIL5x2gd1yuDMCfo5EG","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332756.160613_104.1","section":"bucket","name":"ajxnhd-44","timestamp":"2020-05-24 15:05:56.160613150Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HjB0J3-_3Ecbwb6E9e1-pSe","ver":1},"status":{"status":"write"}}},{"id":"1_1590332756.290427_105.1","section":"bucket","name":"ajxnhd-44","timestamp":"2020-05-24 15:05:56.290427832Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_HjB0J3-_3Ecbwb6E9e1-pSe","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332757.010124_106.1","section":"bucket.instance","name":"ajxnhd-45:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2","timestamp":"2020-05-24 15:05:57.010124087Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_msmvDcyWLYbRK7uS3W_TAqO","ver":1},"status":{"status":"write"}}},{"id":"1_1590332757.154995_107.1","section":"bucket.instance","name":"ajxnhd-45:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2","timestamp":"2020-05-24 15:05:57.154995444Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_msmvDcyWLYbRK7uS3W_TAqO","ver":1},"status":{"status":"complete"}}},{"id":"1_1590332757.220737_108.1","section":"bucket","name":"ajxnhd-45","timestamp":"2020-05-24 15:05:57.220737344Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-Be1tdZHgEt-kqvJvs319g7","ver":1},"status":{"status":"write"}}},{"id":"1_1590332757.336629_109.1","section":"bucket","name":"ajxnhd-45","timestamp":"2020-05-24 15:05:57.336629140Z","data":{"read_version":{"tag":"","ver":0},"write_version":{"tag":"_-Be1tdZHgEt-kqvJvs319g7","ver":1},"status":{"status":"complete"}}}] 2020-05-24T15:07:18.168 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:18.169 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:18.170 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:07:22.411 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:22.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:22.412 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:07:26.847 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:26.848 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:26.848 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:07:29.753 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:29.754 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:07:29.755 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:07:32.960 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:32.960 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:32.961 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 114a3cfc-32a6-4ae9-bab5-baac4f59a79e --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:36.125 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2020-05-24T15:07:36.125 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:36.126 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:36.127 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 114a3cfc-32a6-4ae9-bab5-baac4f59a79e --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:36.326 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:19:10.401 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:07:36.334 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:19:10.408 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:07:36.558 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:19:10.632 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:07:38.716 INFO:teuthology.orchestra.run.smithi160.stdout:[] 2020-05-24T15:07:38.716 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:38.717 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:38.717 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 114a3cfc-32a6-4ae9-bab5-baac4f59a79e --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:41.443 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2020-05-24T15:07:41.443 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:41.444 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '114a3cfc-32a6-4ae9-bab5-baac4f59a79e', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:41.445 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 114a3cfc-32a6-4ae9-bab5-baac4f59a79e --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:43.935 INFO:teuthology.orchestra.run.smithi160.stdout:[] 2020-05-24T15:07:43.935 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:43.936 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:43.936 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 7920f6a5-aa4e-4bd1-aca6-86793c8b287f --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:47.033 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2020-05-24T15:07:47.034 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:47.034 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:47.035 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 7920f6a5-aa4e-4bd1-aca6-86793c8b287f --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:49.890 INFO:teuthology.orchestra.run.smithi160.stdout:[] 2020-05-24T15:07:49.890 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:49.891 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:49.892 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 7920f6a5-aa4e-4bd1-aca6-86793c8b287f --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:53.017 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2020-05-24T15:07:53.018 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:53.019 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '7920f6a5-aa4e-4bd1-aca6-86793c8b287f', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:53.019 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 7920f6a5-aa4e-4bd1-aca6-86793c8b287f --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:55.801 INFO:teuthology.orchestra.run.smithi160.stdout:[] 2020-05-24T15:07:55.801 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:55.802 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:55.802 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period f35208d4-7215-4668-86dd-a246cde34636 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:07:58.399 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2020-05-24T15:07:58.400 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:58.401 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:07:58.402 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period f35208d4-7215-4668-86dd-a246cde34636 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:01.352 INFO:teuthology.orchestra.run.smithi160.stdout:[] 2020-05-24T15:08:01.352 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:01.353 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:01.354 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period f35208d4-7215-4668-86dd-a246cde34636 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:03.989 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2020-05-24T15:08:03.990 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:03.990 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'f35208d4-7215-4668-86dd-a246cde34636', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:03.991 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period f35208d4-7215-4668-86dd-a246cde34636 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:06.383 INFO:teuthology.orchestra.run.smithi160.stdout:[] 2020-05-24T15:08:06.384 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_multi_period_incremental_sync ... ok 2020-05-24T15:08:06.386 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-46 2020-05-24T15:08:07.254 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-47 2020-05-24T15:08:08.720 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-48 2020-05-24T15:08:14.856 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:14.856 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:14.857 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:20.821 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590332898.011480_321.1","last_update":"2020-05-24 15:08:18.011480Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:08:20.821 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2020-05-24T15:08:20.822 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:20.822 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:20.823 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:23.556 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3801484488559056654},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590332898.011480_321.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:18.011480Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:08:23.557 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2020-05-24T15:08:23.558 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:23.558 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:23.559 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:26.718 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590332898.011480_321.1","last_update":"2020-05-24 15:08:18.011480Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:08:26.719 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2020-05-24T15:08:26.720 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:26.720 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:26.721 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:29.411 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":14493924251105887636},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590332898.011480_321.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:18.011480Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:08:29.411 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2020-05-24T15:08:29.412 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:29.412 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:29.413 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:32.244 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590332898.011480_321.1","last_update":"2020-05-24 15:08:18.011480Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:08:32.244 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2020-05-24T15:08:32.245 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:32.246 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:32.246 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:35.729 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":12983540485021103314},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590332898.011480_321.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:18.011480Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:08:35.730 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2020-05-24T15:08:35.731 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:35.731 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:35.732 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:38.254 INFO:teuthology.orchestra.run.smithi160.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590332897.444716_320.1","last_update":"2020-05-24 15:08:17.444716Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:08:38.255 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2020-05-24T15:08:38.256 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:38.256 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:38.257 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:41.120 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":10160595564215479999},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590332897.444716_320.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:17.444716Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:08:41.121 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2020-05-24T15:08:41.122 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:41.122 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:41.123 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:43.686 INFO:teuthology.orchestra.run.smithi160.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590332897.444716_320.1","last_update":"2020-05-24 15:08:17.444716Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:08:43.687 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2020-05-24T15:08:43.688 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:43.688 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:43.689 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:46.836 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":5033135909397636583},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590332897.444716_320.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:17.444716Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:08:46.837 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2020-05-24T15:08:46.837 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:46.838 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:46.838 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:46.842 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:19:14.079 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:08:46.847 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:19:14.083 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:08:47.061 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:19:14.299 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:08:49.940 INFO:teuthology.orchestra.run.smithi160.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590332897.444716_320.1","last_update":"2020-05-24 15:08:17.444716Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:08:49.940 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone2 2020-05-24T15:08:49.941 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:49.941 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:49.942 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:08:52.860 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:08:52.860 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:08:57.861 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:57.862 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:08:57.863 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:01.238 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:01.238 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:06.240 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:06.241 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:06.242 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:09.454 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:09.455 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:14.456 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:14.457 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:14.458 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:17.852 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:17.852 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:22.854 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:22.855 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:22.855 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:26.389 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:26.390 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:31.391 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:31.392 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:31.392 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:34.251 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:34.252 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:39.253 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:39.254 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:39.255 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:42.523 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:42.524 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:47.525 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:47.526 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:47.527 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:50.688 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:50.689 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:09:55.691 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:55.692 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:09:55.693 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:09:59.047 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:09:59.048 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:04.050 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:04.051 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:04.052 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:07.224 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:07.225 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:12.226 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:12.227 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:12.228 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:15.451 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:15.452 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:20.453 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:20.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:20.455 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:23.960 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:23.960 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:28.962 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:28.963 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:28.964 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:32.092 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:32.093 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:37.095 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:37.096 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:37.096 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:40.467 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:40.468 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:45.470 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:45.471 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:45.471 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:48.680 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:48.681 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:10:53.683 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:53.684 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:10:53.685 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:10:56.858 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:10:56.858 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:01.860 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:01.862 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:01.862 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:04.823 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:04.824 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:09.825 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:09.826 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:09.827 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:12.970 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:12.971 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:17.973 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:17.974 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:17.975 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:21.368 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:21.368 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:26.370 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:26.371 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:26.371 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:29.495 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:29.495 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:34.497 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:34.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:34.498 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:38.046 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:38.047 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:43.048 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:43.049 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:43.062 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:46.362 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:46.363 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:51.365 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:51.370 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:51.371 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:11:54.679 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:11:54.680 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:11:59.681 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:59.681 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:11:59.682 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:02.908 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:02.908 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:07.909 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:07.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:07.911 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:11.402 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:11.403 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:16.410 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:16.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:16.412 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:19.744 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:19.744 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:24.745 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:24.746 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:24.747 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:28.046 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:28.047 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:32.754 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:24:06.827 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:12:32.762 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:24:06.836 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:12:32.932 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:24:07.007 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:12:33.049 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:33.049 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:33.050 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:36.372 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:36.373 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:41.374 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:41.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:41.376 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:44.887 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:44.888 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:49.889 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:49.890 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:49.891 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:12:53.058 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:12:53.059 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:12:58.060 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:58.062 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:12:58.062 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:01.208 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:01.209 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:06.212 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:06.213 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:06.214 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:09.970 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:09.971 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:14.972 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:14.973 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:14.974 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:18.488 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:18.489 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:23.490 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:23.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:23.492 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:26.644 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:26.645 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:31.647 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:31.648 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:31.649 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:34.810 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:34.811 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:37.994 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:24:05.231 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:13:38.004 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:24:05.241 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:13:38.155 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:24:05.392 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:13:39.812 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:39.813 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:39.814 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:42.918 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:42.919 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:47.920 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:47.921 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:47.922 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:51.516 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:51.517 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:13:56.517 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:56.519 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:13:56.519 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:13:59.813 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:13:59.814 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:04.815 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:04.816 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:04.817 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:08.354 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:08.355 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:13.356 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:13.357 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:13.358 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:16.924 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:16.925 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:21.926 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:21.927 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:21.928 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:25.351 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:25.352 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:30.353 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:30.354 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:30.354 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:30.950 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:26:05.024 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:14:30.961 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:26:05.035 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:14:31.124 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:26:05.199 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:14:33.556 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:33.556 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:38.558 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:38.559 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:38.560 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:41.747 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:41.748 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:46.750 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:46.751 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:46.752 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:50.364 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:50.365 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:14:55.367 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:55.368 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:14:55.369 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:14:58.876 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:14:58.876 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:03.877 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:03.878 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:03.879 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:07.647 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:07.648 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:12.649 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:12.650 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:12.651 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:16.469 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:16.469 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:21.471 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:21.472 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:21.473 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:25.429 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:25.430 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:30.431 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:30.433 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:30.434 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:33.676 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:33.677 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:38.678 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:38.679 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:38.680 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:42.408 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:42.409 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:43.307 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:26:10.544 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:15:43.315 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:26:10.552 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:15:43.455 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:26:10.693 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:15:47.410 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:47.412 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:47.413 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:50.823 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:50.824 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:15:55.825 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:55.826 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:15:55.827 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:15:59.466 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:15:59.467 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:04.469 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:04.471 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:04.472 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:07.784 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:07.785 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:12.786 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:12.787 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:12.788 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:16.154 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:16.155 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:21.156 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:21.157 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:21.157 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:24.393 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:24.394 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:29.395 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:29.397 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:29.398 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:32.750 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:32.751 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:37.752 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:37.753 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:37.754 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:41.131 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:41.132 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:46.134 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:46.135 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:46.135 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:49.443 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:49.443 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:16:54.445 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:54.446 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:16:54.446 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:16:58.198 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:16:58.199 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:17:03.199 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:03.200 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:17:03.201 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:17:06.686 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:17:06.687 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:17:11.689 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_datalog_autotrim ... FAIL 2020-05-24T15:17:11.690 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:17:11.690 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:17:11.691 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=false --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:17:12.225 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:17:12.220 7fef22634840 0 failed reading obj info from .rgw.root:zone_info.88888d62-7b54-4b85-bd95-327afc14fb31: (2) No such file or directory 2020-05-24T15:17:12.225 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:17:12.220 7fef22634840 0 WARNING: could not read zone params for zone id=88888d62-7b54-4b85-bd95-327afc14fb31 name=test-zone3 2020-05-24T15:17:12.265 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:17:12.260 7fef22634840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T15:17:12.266 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:17:12.260 7fef22634840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T15:17:12.446 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:17:12.446 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:17:12.447 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:17:12.448 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:17:15.395 INFO:teuthology.orchestra.run.smithi160.stdout:Sending period to new master zone d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T15:17:20.119 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":2,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"false","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:17:20.120 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to false 2020-05-24T15:18:20.122 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-49 2020-05-24T15:18:21.565 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:21.566 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:21.566 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:23.534 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=f35208d4-7215-4668-86dd-a246cde34636 2020-05-24T15:18:24.342 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590333500.941666_142.1","last_update":"2020-05-24 15:18:20.941666Z"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:18:24.343 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:18:24.343 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:24.344 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:24.345 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:27.135 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332889.990779_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:09.990779Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:18:27.136 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1590333500.941666_142.1 target=1_1590332889.990779_125.1 2020-05-24T15:18:32.138 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:32.139 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:32.140 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:34.477 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332889.990779_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:09.990779Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:18:34.478 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1590333500.941666_142.1 target=1_1590332889.990779_125.1 2020-05-24T15:18:39.480 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:39.481 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:39.482 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:42.420 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332889.990779_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:09.990779Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:18:42.420 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1590333500.941666_142.1 target=1_1590332889.990779_125.1 2020-05-24T15:18:47.421 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:47.422 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:47.423 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:50.175 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332889.990779_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:09.990779Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:18:50.176 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1590333500.941666_142.1 target=1_1590332889.990779_125.1 2020-05-24T15:18:55.178 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:55.179 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:55.180 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:18:57.989 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590333500.941666_142.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:20.941666Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:18:57.989 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:18:57.990 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:18:57.990 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:57.991 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:18:57.992 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:00.890 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332889.990779_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:09.990779Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:00.891 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1590333500.941666_142.1 target=1_1590332889.990779_125.1 2020-05-24T15:19:05.894 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:05.895 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:05.896 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:08.662 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332889.990779_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:09.990779Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:08.663 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1590333500.941666_142.1 target=1_1590332889.990779_125.1 2020-05-24T15:19:13.664 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:13.670 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:13.671 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:16.539 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590332889.990779_125.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:08:09.990779Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:16.539 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1590333500.941666_142.1 target=1_1590332889.990779_125.1 2020-05-24T15:19:21.541 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:21.542 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:21.543 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:24.221 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590333500.941666_142.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:20.941666Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:24.222 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:19:24.222 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:19:24.223 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:24.223 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:19:24.224 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:19:27.470 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590333500.941666_142.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:20.941666Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:19:27.471 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:19:27.693 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=d058c7d4-c509-44b6-8645-91a502a0fccb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:19:27.694 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=d058c7d4-c509-44b6-8645-91a502a0fccb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:19:27.695 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone=d058c7d4-c509-44b6-8645-91a502a0fccb --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:19:28.093 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:19:28.086 7f872cc0d840 0 failed reading obj info from .rgw.root:zone_info.88888d62-7b54-4b85-bd95-327afc14fb31: (2) No such file or directory 2020-05-24T15:19:28.093 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:19:28.086 7f872cc0d840 0 WARNING: could not read zone params for zone id=88888d62-7b54-4b85-bd95-327afc14fb31 name=test-zone3 2020-05-24T15:19:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:19:28.110 7f872cc0d840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T15:19:28.115 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:19:28.110 7f872cc0d840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T15:19:28.230 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:19:28.231 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:19:28.232 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:19:28.232 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:19:30.977 INFO:teuthology.orchestra.run.smithi160.stdout:Sending period to new master zone d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T15:19:34.463 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":3,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"false","sync_from":[],"redirect_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb"},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:19:34.464 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "d058c7d4-c509-44b6-8645-91a502a0fccb" 2020-05-24T15:20:37.103 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:20:37.105 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:20:37.106 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=true --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:20:37.716 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:20:37.712 7fa398279840 0 failed reading obj info from .rgw.root:zone_info.88888d62-7b54-4b85-bd95-327afc14fb31: (2) No such file or directory 2020-05-24T15:20:37.717 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:20:37.712 7fa398279840 0 WARNING: could not read zone params for zone id=88888d62-7b54-4b85-bd95-327afc14fb31 name=test-zone3 2020-05-24T15:20:37.746 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:20:37.742 7fa398279840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T15:20:37.746 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:20:37.742 7fa398279840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T15:20:37.860 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:20:37.861 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:20:37.861 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:20:37.862 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:20:40.540 INFO:teuthology.orchestra.run.smithi160.stdout:Sending period to new master zone d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T15:20:43.855 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":4,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb"},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:20:43.856 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to true 2020-05-24T15:20:53.498 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:32:27.572 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:20:53.510 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:32:27.583 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:20:53.696 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:32:27.771 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:26.606 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:33:00.681 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:26.622 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:33:00.693 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:26.743 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:33:00.817 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:43.858 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:21:43.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:21:43.860 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone= --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:21:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:21:44.389 7fbef36ec840 0 failed reading obj info from .rgw.root:zone_info.88888d62-7b54-4b85-bd95-327afc14fb31: (2) No such file or directory 2020-05-24T15:21:44.394 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:21:44.389 7fbef36ec840 0 WARNING: could not read zone params for zone id=88888d62-7b54-4b85-bd95-327afc14fb31 name=test-zone3 2020-05-24T15:21:44.430 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:21:44.425 7fbef36ec840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T15:21:44.431 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:21:44.425 7fbef36ec840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T15:21:44.618 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","domain_root":"test-zone2.rgw.meta:root","control_pool":"test-zone2.rgw.control","gc_pool":"test-zone2.rgw.log:gc","lc_pool":"test-zone2.rgw.log:lc","log_pool":"test-zone2.rgw.log","intent_log_pool":"test-zone2.rgw.log:intent","usage_log_pool":"test-zone2.rgw.log:usage","reshard_pool":"test-zone2.rgw.log:reshard","user_keys_pool":"test-zone2.rgw.meta:users.keys","user_email_pool":"test-zone2.rgw.meta:users.email","user_swift_pool":"test-zone2.rgw.meta:users.swift","user_uid_pool":"test-zone2.rgw.meta:users.uid","otp_pool":"test-zone2.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"test-zone2.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"test-zone2.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"test-zone2.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:21:44.618 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:21:44.619 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:21:44.620 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:21:47.462 INFO:teuthology.orchestra.run.smithi160.stdout:Sending period to new master zone d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T15:21:52.080 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":5,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:21:52.080 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "" 2020-05-24T15:21:58.856 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:32:26.093 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:58.858 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:32:26.095 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:21:59.033 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:32:26.270 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:22:37.623 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:33:04.860 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:22:37.640 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:33:04.877 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:22:37.753 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:33:04.990 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:22:52.082 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_multi_zone_redirect ... ok 2020-05-24T15:22:52.084 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:22:52.085 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:22:52.085 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:22:54.314 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=f35208d4-7215-4668-86dd-a246cde34636 2020-05-24T15:22:55.181 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590333500.941666_142.1","last_update":"2020-05-24 15:18:20.941666Z"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:22:55.182 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:22:55.182 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:22:55.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:22:55.184 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:22:57.919 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590333500.941666_142.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:20.941666Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:22:57.920 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:22:57.920 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:22:57.921 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:22:57.921 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:22:57.922 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:23:00.775 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590333500.941666_142.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:20.941666Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:23:00.776 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:23:00.776 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:23:00.777 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:00.777 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:23:00.778 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:23:04.291 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590333500.941666_142.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:18:20.941666Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:23:04.292 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:23:04.292 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:04.293 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'UBR0JQ2W32DCDP87', '--secret', 'T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:04.294 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key UBR0JQ2W32DCDP87 --secret T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:23:05.612 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:05.608 7ffb2f996840 0 failed reading obj info from .rgw.root:zone_info.88888d62-7b54-4b85-bd95-327afc14fb31: (2) No such file or directory 2020-05-24T15:23:05.613 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:05.608 7ffb2f996840 0 WARNING: could not read zone params for zone id=88888d62-7b54-4b85-bd95-327afc14fb31 name=test-zone3 2020-05-24T15:23:05.636 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:05.633 7ffb2f996840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T15:23:05.637 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:05.633 7ffb2f996840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T15:23:05.715 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"fe227a21-9703-443e-a35f-7040c61e31a8","name":"remove","domain_root":"remove.rgw.meta:root","control_pool":"remove.rgw.control","gc_pool":"remove.rgw.log:gc","lc_pool":"remove.rgw.log:lc","log_pool":"remove.rgw.log","intent_log_pool":"remove.rgw.log:intent","usage_log_pool":"remove.rgw.log:usage","reshard_pool":"remove.rgw.log:reshard","user_keys_pool":"remove.rgw.meta:users.keys","user_email_pool":"remove.rgw.meta:users.email","user_swift_pool":"remove.rgw.meta:users.swift","user_uid_pool":"remove.rgw.meta:users.uid","otp_pool":"remove.rgw.otp","system_key":{"access_key":"UBR0JQ2W32DCDP87","secret_key":"T5UzuIN1lx4zenHdMVNtlfM19VmcRiKa"},"placement_pools":[{"key":"default-placement","val":{"index_pool":"remove.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"remove.rgw.buckets.data"}},"data_extra_pool":"remove.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:23:05.716 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:05.717 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:05.717 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:23:46.954 INFO:teuthology.orchestra.run.smithi160.stdout:Sending period to new master zone d058c7d4-c509-44b6-8645-91a502a0fccb 2020-05-24T15:23:50.170 INFO:teuthology.orchestra.run.smithi160.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":6,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"fe227a21-9703-443e-a35f-7040c61e31a8","name":"remove","endpoints":[],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755},{"key":"fe227a21-9703-443e-a35f-7040c61e31a8","val":3483842831}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:23:50.171 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:50.172 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:50.173 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:23:50.482 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:23:50.481 7f0e9d312840 0 failed reading obj info from .rgw.root:zone_info.1a48d5d2-db65-4e7e-9296-4bd2463c8a06: (2) No such file or directory 2020-05-24T15:23:50.483 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:23:50.481 7f0e9d312840 0 WARNING: could not read zone params for zone id=1a48d5d2-db65-4e7e-9296-4bd2463c8a06 name=test-zone4 2020-05-24T15:23:50.526 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:23:50.524 7f0e9d312840 0 failed reading obj info from .rgw.root:zone_info.8de99cfe-4678-427d-bca8-2f77415e7812: (2) No such file or directory 2020-05-24T15:23:50.526 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:23:50.524 7f0e9d312840 0 WARNING: could not read zone params for zone id=8de99cfe-4678-427d-bca8-2f77415e7812 name=test-zone2 2020-05-24T15:23:50.605 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:23:50.606 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:50.607 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:50.607 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:23:50.870 INFO:teuthology.orchestra.run.smithi050.stderr:zone name remove not found in zonegroup test-zonegroup 2020-05-24T15:23:50.875 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-05-24T15:23:50.876 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:50.877 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:50.877 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone delete --rgw-zone remove --cluster c2 --debug-rgw 0 --debug-ms 0 2020-05-24T15:23:51.895 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:51.891 7f4b3658f840 0 failed reading obj info from .rgw.root:zone_info.88888d62-7b54-4b85-bd95-327afc14fb31: (2) No such file or directory 2020-05-24T15:23:51.896 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:51.891 7f4b3658f840 0 WARNING: could not read zone params for zone id=88888d62-7b54-4b85-bd95-327afc14fb31 name=test-zone3 2020-05-24T15:23:51.921 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:51.917 7f4b3658f840 0 failed reading obj info from .rgw.root:zone_info.d058c7d4-c509-44b6-8645-91a502a0fccb: (2) No such file or directory 2020-05-24T15:23:51.922 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:51.917 7f4b3658f840 0 WARNING: could not read zone params for zone id=d058c7d4-c509-44b6-8645-91a502a0fccb name=test-zone1 2020-05-24T15:23:52.032 INFO:teuthology.orchestra.run.smithi160.stderr:2020-05-24 15:23:52.027 7f4b3658f840 0 zone id fe227a21-9703-443e-a35f-7040c61e31a8 is not a part of zonegroup default 2020-05-24T15:23:52.244 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:52.245 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:23:52.245 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:23:55.406 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:23:55.405 7f74f85b2840 0 WARNING: can't generate connection for zone fe227a21-9703-443e-a35f-7040c61e31a8 id remove: no endpoints defined 2020-05-24T15:24:01.970 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":7,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:24:01.971 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_zonegroup_remove ... ok 2020-05-24T15:24:01.972 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:01.973 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:01.974 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup create --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:02.677 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"14c0c33a-b59d-4fc8-b765-62c9eb04ab12","name":"remove_zg","api_name":"remove_zg","is_master":"false","endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"","zones":[],"placement_targets":[],"default_placement":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:24:02.678 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:02.678 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:02.679 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:04.648 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"74229baa-e2ba-4e32-b8af-cd790450f248","name":"remove","domain_root":"remove.rgw.meta:root","control_pool":"remove.rgw.control","gc_pool":"remove.rgw.log:gc","lc_pool":"remove.rgw.log:lc","log_pool":"remove.rgw.log","intent_log_pool":"remove.rgw.log:intent","usage_log_pool":"remove.rgw.log:usage","reshard_pool":"remove.rgw.log:reshard","user_keys_pool":"remove.rgw.meta:users.keys","user_email_pool":"remove.rgw.meta:users.email","user_swift_pool":"remove.rgw.meta:users.swift","user_uid_pool":"remove.rgw.meta:users.uid","otp_pool":"remove.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"remove.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"remove.rgw.buckets.data"}},"data_extra_pool":"remove.rgw.buckets.non-ec","index_type":0}}],"metadata_heap":"","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"} 2020-05-24T15:24:04.649 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:04.650 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:04.651 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:10.586 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":8,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"14c0c33a-b59d-4fc8-b765-62c9eb04ab12","name":"remove_zg","api_name":"remove_zg","is_master":"false","endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"74229baa-e2ba-4e32-b8af-cd790450f248","zones":[{"id":"74229baa-e2ba-4e32-b8af-cd790450f248","name":"remove","endpoints":[],"log_meta":"false","log_data":"false","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"},{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"74229baa-e2ba-4e32-b8af-cd790450f248","val":3513408866},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:24:10.587 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:10.588 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:10.589 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone delete --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:11.889 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:24:11.887 7efd238e2840 0 zone id 74229baa-e2ba-4e32-b8af-cd790450f248 is not a part of zonegroup test-zonegroup 2020-05-24T15:24:11.926 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:24:11.925 7efd238e2840 0 zone id 74229baa-e2ba-4e32-b8af-cd790450f248 is not a part of zonegroup default 2020-05-24T15:24:12.140 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:12.141 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:12.142 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:13.502 INFO:teuthology.orchestra.run.smithi050.stderr:2020-05-24 15:24:13.500 7f60c8a29840 0 ERROR: zonegroup remove_zg has a non existent master zone 2020-05-24T15:24:13.506 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-05-24T15:24:13.507 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:13.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:13.508 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup delete --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:14.081 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:14.082 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:24:14.082 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:24:19.856 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"f35208d4-7215-4668-86dd-a246cde34636","epoch":9,"predecessor_uuid":"7920f6a5-aa4e-4bd1-aca6-86793c8b287f","sync_status":["","","1_1590332684.147703_160.1",""],"period_map":{"id":"f35208d4-7215-4668-86dd-a246cde34636","zonegroups":[{"id":"test-zonegroup","name":"test-zonegroup","api_name":"","is_master":"true","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"hostnames":[],"hostnames_s3website":[],"master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","zones":[{"id":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","name":"test-zone4","endpoints":["http://smithi160.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"pubsub","sync_from_all":"false","sync_from":["test-zone1"],"redirect_zone":""},{"id":"88888d62-7b54-4b85-bd95-327afc14fb31","name":"test-zone3","endpoints":["http://smithi050.front.sepia.ceph.com:8001"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"8de99cfe-4678-427d-bca8-2f77415e7812","name":"test-zone2","endpoints":["http://smithi160.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""},{"id":"d058c7d4-c509-44b6-8645-91a502a0fccb","name":"test-zone1","endpoints":["http://smithi050.front.sepia.ceph.com:8000"],"log_meta":"false","log_data":"true","bucket_index_max_shards":0,"read_only":"false","tier_type":"","sync_from_all":"true","sync_from":[],"redirect_zone":""}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d"}],"short_zone_ids":[{"key":"1a48d5d2-db65-4e7e-9296-4bd2463c8a06","val":1936634423},{"key":"88888d62-7b54-4b85-bd95-327afc14fb31","val":3263476411},{"key":"8de99cfe-4678-427d-bca8-2f77415e7812","val":3083873520},{"key":"d058c7d4-c509-44b6-8645-91a502a0fccb","val":193387755}]},"master_zonegroup":"test-zonegroup","master_zone":"d058c7d4-c509-44b6-8645-91a502a0fccb","period_config":{"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}},"realm_id":"96d5f8b6-5067-4aa4-9df8-9381cd7a4c4d","realm_name":"test-realm","realm_epoch":4} 2020-05-24T15:24:20.048 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_zg_master_zone_delete ... ok 2020-05-24T15:24:20.049 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-50 2020-05-24T15:24:33.328 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-51 2020-05-24T15:24:36.305 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-52 2020-05-24T15:24:39.295 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-53 2020-05-24T15:24:40.635 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-54 2020-05-24T15:25:01.548 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-55 2020-05-24T15:25:10.674 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_set_bucket_website ... tasks.rgw_multi.tests.test_set_bucket_policy ... ok 2020-05-24T15:25:10.676 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-56 2020-05-24T15:25:11.746 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-57 2020-05-24T15:25:13.159 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-58 2020-05-24T15:25:14.880 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'ajxnhd-56', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:14.882 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'ajxnhd-56', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:14.883 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket ajxnhd-56 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:25:18.597 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'ajxnhd-57', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:18.598 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'ajxnhd-57', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:18.599 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket ajxnhd-57 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:25:21.930 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'ajxnhd-58', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:21.930 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'ajxnhd-58', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2020-05-24T15:25:21.931 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket ajxnhd-58 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2020-05-24T15:25:25.259 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:25.261 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:25.262 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 log list --rgw-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:28.399 INFO:teuthology.orchestra.run.smithi050.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","obj_delete_at_hint.0000000028","obj_delete_at_hint.0000000040","meta.full-sync.index.0","sync.error-log.6","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18-ajxnhd-9","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-46:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32-ajxnhd-23","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-48:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35-ajxnhd-26","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.0","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","data.full-sync.index.8de99cfe-4678-427d-bca8-2f77415e7812.3","obj_delete_at_hint.0000000096","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41-ajxnhd-32","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.6-ajxnhd-55","obj_delete_at_hint.0000000006","data.full-sync.index.88888d62-7b54-4b85-bd95-327afc14fb31.2","obj_delete_at_hint.0000000029","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.2.retry","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.3-ajxnhd-52","bilog.trim","obj_delete_at_hint.0000000016","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4-ajxnhd-47","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22-ajxnhd-13","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1-ajxnhd-1","obj_delete_at_hint.0000000092","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4-ajxnhd-4","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3-ajxnhd-46","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28-ajxnhd-19","sync.error-log.4","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.9-ajxnhd-58","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","obj_delete_at_hint.0000000014","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38-ajxnhd-29","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21-ajxnhd-12","obj_delete_at_hint.0000000064","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.1.retry","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19-ajxnhd-10","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40-ajxnhd-31","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","data_log.1","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.0","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33-ajxnhd-24","meta.full-sync.index.1","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45-ajxnhd-36","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-47:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26-ajxnhd-17","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46-ajxnhd-37","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48-ajxnhd-39","obj_delete_at_hint.0000000053","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","obj_delete_at_hint.0000000091","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.4-ajxnhd-53","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21-ajxnhd-12","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24-ajxnhd-15","sync.error-log.3","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20-ajxnhd-11","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","mdlog.sync-status.shard.2","obj_delete_at_hint.0000000045","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2-ajxnhd-2","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19-ajxnhd-10","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44-ajxnhd-35","mdlog.sync-status.shard.1","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2-ajxnhd-45","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.36-ajxnhd-27","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","datalog.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812","obj_delete_at_hint.0000000066","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-49:d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1","obj_delete_at_hint.0000000068","data.full-sync.index.8de99cfe-4678-427d-bca8-2f77415e7812.1","obj_delete_at_hint.0000000085","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.2","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","2020-05-24-15-8de99cfe-4678-427d-bca8-2f77415e7812.8073.1-ajxnhd-42","mdlog.sync-status.shard.3","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27-ajxnhd-18","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50-ajxnhd-41","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","mdlog.sync-status.shard.0","obj_delete_at_hint.0000000027","mdlog.sync-status","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.3","obj_delete_at_hint.0000000019","data.full-sync.index.88888d62-7b54-4b85-bd95-327afc14fb31.3","obj_delete_at_hint.0000000039","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","obj_delete_at_hint.0000000100","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3-ajxnhd-3","obj_delete_at_hint.0000000093","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5-ajxnhd-5","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47-ajxnhd-38","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.2-ajxnhd-51","obj_delete_at_hint.0000000057","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.1","data.full-sync.index.88888d62-7b54-4b85-bd95-327afc14fb31.1","2020-05-24-15-8de99cfe-4678-427d-bca8-2f77415e7812.8073.2-ajxnhd-43","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","sync.error-log.2","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-48:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5","obj_delete_at_hint.0000000012","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","datalog.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.5-ajxnhd-54","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37-ajxnhd-28","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.8-ajxnhd-57","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","obj_delete_at_hint.0000000111","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17-ajxnhd-8","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23-ajxnhd-14","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.7-ajxnhd-56","obj_delete_at_hint.0000000072","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30-ajxnhd-21","data.full-sync.index.8de99cfe-4678-427d-bca8-2f77415e7812.2","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","sync.error-log.1","obj_delete_at_hint.0000000033","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1-ajxnhd-44","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.1.retry","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20-ajxnhd-11","obj_delete_at_hint.0000000108","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25-ajxnhd-16","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","data.full-sync.index.88888d62-7b54-4b85-bd95-327afc14fb31.0","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.1-ajxnhd-50","data.full-sync.index.8de99cfe-4678-427d-bca8-2f77415e7812.0","obj_delete_at_hint.0000000051","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","obj_delete_at_hint.0000000106","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.42-ajxnhd-33","obj_delete_at_hint.0000000017","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-47:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4","obj_delete_at_hint.0000000023","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","meta.full-sync.index.2","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29-ajxnhd-20","obj_delete_at_hint.0000000067","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.3","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-49:d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1","obj_delete_at_hint.0000000086","meta.log.f35208d4-7215-4668-86dd-a246cde34636.2","sync.error-log.5","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16-ajxnhd-7","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.3.retry","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-46:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43-ajxnhd-34","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.0.retry","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.1","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","obj_delete_at_hint.0000000124","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.39-ajxnhd-30","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49-ajxnhd-40","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31-ajxnhd-22","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1-ajxnhd-49","obj_delete_at_hint.0000000055","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.2","obj_delete_at_hint.0000000061","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34-ajxnhd-25","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5-ajxnhd-48","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6-ajxnhd-6","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22"] 2020-05-24T15:25:29.039 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:29.040 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:29.041 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:32.769 INFO:teuthology.orchestra.run.smithi160.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","obj_delete_at_hint.0000000028","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","obj_delete_at_hint.0000000040","meta.full-sync.index.0","sync.error-log.6","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","datalog.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.1","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18-ajxnhd-9","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-46:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32-ajxnhd-23","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35-ajxnhd-26","obj_delete_at_hint.0000000076","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.0","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","obj_delete_at_hint.0000000006","data.full-sync.index.88888d62-7b54-4b85-bd95-327afc14fb31.2","obj_delete_at_hint.0000000029","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.2.retry","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.1","bilog.trim","obj_delete_at_hint.0000000016","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4-ajxnhd-47","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22-ajxnhd-13","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-46:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1-ajxnhd-1","obj_delete_at_hint.0000000092","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4-ajxnhd-4","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3-ajxnhd-46","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28-ajxnhd-19","sync.error-log.4","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21-ajxnhd-12","obj_delete_at_hint.0000000064","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19-ajxnhd-10","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.40-ajxnhd-31","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-47:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","data_log.1","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.3","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.0","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33-ajxnhd-24","meta.full-sync.index.1","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45-ajxnhd-36","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26-ajxnhd-17","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46-ajxnhd-37","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48-ajxnhd-39","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.3","obj_delete_at_hint.0000000053","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21-ajxnhd-12","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.1.retry","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24-ajxnhd-15","sync.error-log.3","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20-ajxnhd-11","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","mdlog.sync-status.shard.2","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.0","obj_delete_at_hint.0000000045","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2-ajxnhd-2","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19-ajxnhd-10","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","mdlog.sync-status.shard.1","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2-ajxnhd-45","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-49:d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1","obj_delete_at_hint.0000000073","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.2","obj_delete_at_hint.0000000038","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","2020-05-24-15-8de99cfe-4678-427d-bca8-2f77415e7812.8073.1-ajxnhd-42","mdlog.sync-status.shard.3","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50-ajxnhd-41","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","mdlog.sync-status.shard.0","obj_delete_at_hint.0000000027","mdlog.sync-status","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.3","obj_delete_at_hint.0000000019","data.full-sync.index.88888d62-7b54-4b85-bd95-327afc14fb31.3","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3-ajxnhd-3","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","obj_delete_at_hint.0000000093","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5-ajxnhd-5","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47-ajxnhd-38","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.2-ajxnhd-51","obj_delete_at_hint.0000000057","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.1","data.full-sync.index.88888d62-7b54-4b85-bd95-327afc14fb31.1","2020-05-24-15-8de99cfe-4678-427d-bca8-2f77415e7812.8073.2-ajxnhd-43","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","sync.error-log.2","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-48:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5","obj_delete_at_hint.0000000012","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","datalog.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.5-ajxnhd-54","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.37-ajxnhd-28","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.8-ajxnhd-57","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","obj_delete_at_hint.0000000111","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17-ajxnhd-8","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.2","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23-ajxnhd-14","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","obj_delete_at_hint.0000000072","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30-ajxnhd-21","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","sync.error-log.1","obj_delete_at_hint.0000000033","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1-ajxnhd-44","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.1.retry","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20-ajxnhd-11","obj_delete_at_hint.0000000108","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25-ajxnhd-16","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","data.full-sync.index.88888d62-7b54-4b85-bd95-327afc14fb31.0","obj_delete_at_hint.0000000051","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","obj_delete_at_hint.0000000106","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-48:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5","obj_delete_at_hint.0000000017","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-47:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","obj_delete_at_hint.0000000023","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","meta.full-sync.index.2","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29-ajxnhd-20","obj_delete_at_hint.0000000067","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-49:d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1","obj_delete_at_hint.0000000086","meta.log.f35208d4-7215-4668-86dd-a246cde34636.2","sync.error-log.5","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16-ajxnhd-7","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.3.retry","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.43-ajxnhd-34","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.0.retry","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49-ajxnhd-40","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31-ajxnhd-22","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1-ajxnhd-49","obj_delete_at_hint.0000000055","datalog.sync-status.shard.88888d62-7b54-4b85-bd95-327afc14fb31.2","obj_delete_at_hint.0000000061","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34-ajxnhd-25","obj_delete_at_hint.0000000097","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","obj_delete_at_hint.0000000042","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5-ajxnhd-48","bucket.sync-status.88888d62-7b54-4b85-bd95-327afc14fb31:ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6-ajxnhd-6"] 2020-05-24T15:25:33.323 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['log', 'list', '--rgw-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:33.324 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'log', 'list', '--rgw-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:33.325 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 log list --rgw-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:36.461 INFO:teuthology.orchestra.run.smithi050.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","obj_delete_at_hint.0000000028","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","obj_delete_at_hint.0000000040","meta.full-sync.index.0","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","datalog.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.1","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.18-ajxnhd-9","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32-ajxnhd-23","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-48:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35-ajxnhd-26","obj_delete_at_hint.0000000076","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.0","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.0","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","data.full-sync.index.8de99cfe-4678-427d-bca8-2f77415e7812.3","obj_delete_at_hint.0000000096","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.41-ajxnhd-32","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.6-ajxnhd-55","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.1","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.3-ajxnhd-52","bilog.trim","obj_delete_at_hint.0000000016","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4-ajxnhd-47","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22-ajxnhd-13","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-46:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.1-ajxnhd-1","obj_delete_at_hint.0000000092","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.4-ajxnhd-4","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3-ajxnhd-46","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28-ajxnhd-19","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5442.9-ajxnhd-58","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","obj_delete_at_hint.0000000014","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.38-ajxnhd-29","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21-ajxnhd-12","obj_delete_at_hint.0000000064","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.1.retry","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19-ajxnhd-10","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-47:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","data_log.1","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.3","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33-ajxnhd-24","meta.full-sync.index.1","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45-ajxnhd-36","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-47:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26-ajxnhd-17","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46-ajxnhd-37","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.48-ajxnhd-39","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.3","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.1.retry","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24-ajxnhd-15","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20-ajxnhd-11","mdlog.sync-status.shard.2","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.0","obj_delete_at_hint.0000000045","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.2-ajxnhd-2","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19-ajxnhd-10","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.44-ajxnhd-35","mdlog.sync-status.shard.1","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.2-ajxnhd-45","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","datalog.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812","obj_delete_at_hint.0000000066","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-49:d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1","obj_delete_at_hint.0000000068","data.full-sync.index.8de99cfe-4678-427d-bca8-2f77415e7812.1","obj_delete_at_hint.0000000085","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.2","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-49:d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1","obj_delete_at_hint.0000000073","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.2","obj_delete_at_hint.0000000038","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","meta.log.114a3cfc-32a6-4ae9-bab5-baac4f59a79e.0","obj_delete_at_hint.0000000103","2020-05-24-15-8de99cfe-4678-427d-bca8-2f77415e7812.8073.1-ajxnhd-42","mdlog.sync-status.shard.3","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.27-ajxnhd-18","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.50-ajxnhd-41","mdlog.sync-status.shard.0","obj_delete_at_hint.0000000027","mdlog.sync-status","obj_delete_at_hint.0000000019","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","obj_delete_at_hint.0000000039","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","meta.log.7920f6a5-aa4e-4bd1-aca6-86793c8b287f.2","obj_delete_at_hint.0000000100","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.3-ajxnhd-3","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","obj_delete_at_hint.0000000093","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.5-ajxnhd-5","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47-ajxnhd-38","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","2020-05-24-15-8de99cfe-4678-427d-bca8-2f77415e7812.8073.2-ajxnhd-43","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","obj_delete_at_hint.0000000012","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","obj_delete_at_hint.0000000111","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.17-ajxnhd-8","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.2","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23-ajxnhd-14","obj_delete_at_hint.0000000072","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30-ajxnhd-21","data.full-sync.index.8de99cfe-4678-427d-bca8-2f77415e7812.2","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","obj_delete_at_hint.0000000033","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.1-ajxnhd-44","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20-ajxnhd-11","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25-ajxnhd-16","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","data.full-sync.index.8de99cfe-4678-427d-bca8-2f77415e7812.0","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-48:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5","obj_delete_at_hint.0000000017","meta.log.114a3cfc-32a6-4ae9-bab5-baac4f59a79e.2","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","meta.full-sync.index.2","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29-ajxnhd-20","obj_delete_at_hint.0000000067","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.3","obj_delete_at_hint.0000000086","meta.log.f35208d4-7215-4668-86dd-a246cde34636.2","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.16-ajxnhd-7","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-46:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","datalog.sync-status.shard.8de99cfe-4678-427d-bca8-2f77415e7812.1","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.49-ajxnhd-40","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31-ajxnhd-22","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1-ajxnhd-49","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","2020-05-24-14-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34-ajxnhd-25","obj_delete_at_hint.0000000097","meta.log.114a3cfc-32a6-4ae9-bab5-baac4f59a79e.3","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","obj_delete_at_hint.0000000042","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030","2020-05-24-15-d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5-ajxnhd-48","2020-05-24-13-d058c7d4-c509-44b6-8645-91a502a0fccb.4397.6-ajxnhd-6","bucket.sync-status.8de99cfe-4678-427d-bca8-2f77415e7812:ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22"] 2020-05-24T15:25:37.114 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone4', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:37.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone4', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:37.116 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone4 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:40.444 INFO:teuthology.orchestra.run.smithi160.stdout:["obj_delete_at_hint.0000000078","meta.history","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","obj_delete_at_hint.0000000028","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-36:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.45","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-37:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.46","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-25:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.34","obj_delete_at_hint.0000000040","meta.full-sync.index.0","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","datalog.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-22:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.31","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.1","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","obj_delete_at_hint.0000000076","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.0","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-16:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.25","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.1","bilog.trim","obj_delete_at_hint.0000000016","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-46:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.3","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-47:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.4","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.3","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-17:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.26","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","meta.full-sync.index.1","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-10:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.19","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-19:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.28","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-14:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.23","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.3","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.1.retry","mdlog.sync-status.shard.2","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.0","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","mdlog.sync-status.shard.1","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-49:d058c7d4-c509-44b6-8645-91a502a0fccb.5234.1","obj_delete_at_hint.0000000073","datalog.sync-status.shard.d058c7d4-c509-44b6-8645-91a502a0fccb.2","obj_delete_at_hint.0000000038","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-11:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.20","meta.full-sync.index.3","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","meta.log.114a3cfc-32a6-4ae9-bab5-baac4f59a79e.0","obj_delete_at_hint.0000000103","mdlog.sync-status.shard.3","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-20:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.29","mdlog.sync-status.shard.0","obj_delete_at_hint.0000000027","mdlog.sync-status","obj_delete_at_hint.0000000019","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-21:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.30","obj_delete_at_hint.0000000039","meta.log.7920f6a5-aa4e-4bd1-aca6-86793c8b287f.2","obj_delete_at_hint.0000000100","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-13:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.22","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","obj_delete_at_hint.0000000012","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-26:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.35","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-12:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.21","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-23:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.32","data.full-sync.index.d058c7d4-c509-44b6-8645-91a502a0fccb.2","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-48:d058c7d4-c509-44b6-8645-91a502a0fccb.5056.5","obj_delete_at_hint.0000000017","meta.log.114a3cfc-32a6-4ae9-bab5-baac4f59a79e.2","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-15:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.24","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","meta.full-sync.index.2","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","meta.log.f35208d4-7215-4668-86dd-a246cde34636.2","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-38:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.47","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","meta.log.114a3cfc-32a6-4ae9-bab5-baac4f59a79e.3","bucket.sync-status.d058c7d4-c509-44b6-8645-91a502a0fccb:ajxnhd-24:d058c7d4-c509-44b6-8645-91a502a0fccb.4397.33","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2020-05-24T15:25:41.295 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:41.297 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:41.298 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:43.871 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590333924.599103_488.1","last_update":"2020-05-24 15:25:24.599103Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:25:43.871 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2020-05-24T15:25:43.872 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:43.872 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:43.873 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:46.619 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3801484488559056654},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590333642.058354_395.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:20:42.058354Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:25:46.620 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=1_1590333924.599103_488.1 target=1_1590333642.058354_395.1 2020-05-24T15:25:51.621 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:51.622 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:51.623 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:25:54.147 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3801484488559056654},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590333642.058354_395.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:20:42.058354Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:25:54.148 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=1_1590333924.599103_488.1 target=1_1590333642.058354_395.1 2020-05-24T15:25:59.149 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:59.150 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:25:59.152 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:02.907 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3801484488559056654},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590333642.058354_395.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:20:42.058354Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:02.908 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=1_1590333924.599103_488.1 target=1_1590333642.058354_395.1 2020-05-24T15:26:07.909 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:07.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:07.911 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:10.834 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":3801484488559056654},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590333924.599103_488.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:25:24.599103Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:10.834 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2020-05-24T15:26:10.835 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:10.836 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:10.836 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:13.791 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590333924.599103_488.1","last_update":"2020-05-24 15:25:24.599103Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:26:13.792 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2020-05-24T15:26:13.792 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:13.793 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:13.794 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:16.228 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":14493924251105887636},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590333924.599103_488.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:25:24.599103Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:16.229 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2020-05-24T15:26:16.230 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:16.231 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:16.232 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:18.907 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590333924.599103_488.1","last_update":"2020-05-24 15:25:24.599103Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:26:18.908 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2020-05-24T15:26:18.908 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:18.909 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:18.909 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:22.025 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":12983540485021103314},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590333924.599103_488.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:25:24.599103Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:22.026 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2020-05-24T15:26:22.026 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:22.027 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:22.027 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:24.450 INFO:teuthology.orchestra.run.smithi160.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590333925.255399_481.1","last_update":"2020-05-24 15:25:25.255399Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:26:24.451 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2020-05-24T15:26:24.452 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:24.453 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:24.454 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:27.232 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":10160595564215479999},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590333925.255399_481.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:25:25.255399Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:27.233 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2020-05-24T15:26:27.234 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:27.235 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:27.236 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:30.002 INFO:teuthology.orchestra.run.smithi160.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590333925.255399_481.1","last_update":"2020-05-24 15:25:25.255399Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:26:30.003 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2020-05-24T15:26:30.004 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:30.004 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:30.005 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:32.622 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"instance_id":5033135909397636583},"markers":[{"key":0,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":1,"val":{"status":"incremental-sync","marker":"1_1590333925.255399_481.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:25:25.255399Z"}},{"key":2,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}},{"key":3,"val":{"status":"incremental-sync","marker":"","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"0.000000"}}]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:32.623 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2020-05-24T15:26:32.624 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:32.624 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:32.625 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:35.241 INFO:teuthology.orchestra.run.smithi160.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"1_1590333925.255399_481.1","last_update":"2020-05-24 15:25:25.255399Z"},{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:26:35.242 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone2 2020-05-24T15:26:35.243 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:35.244 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:35.245 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:38.419 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:38.420 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:26:43.423 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:43.424 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:43.425 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:46.769 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:46.770 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:26:51.771 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:51.772 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:26:51.773 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:26:55.253 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:26:55.254 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:00.255 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:00.257 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:00.257 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:03.485 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:03.485 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:08.487 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:08.488 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:08.488 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:11.595 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:11.595 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:16.597 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:16.598 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:16.599 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:19.882 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:19.882 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:24.884 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:24.885 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:24.885 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:28.097 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:28.098 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:33.099 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:33.100 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:33.101 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:36.891 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:36.892 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:41.893 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:41.895 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:41.896 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:45.236 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:45.237 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:50.239 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:50.241 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:50.241 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:27:53.554 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:27:53.555 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:27:58.556 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:58.557 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:27:58.558 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:02.259 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:02.260 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:07.262 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:07.263 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:07.263 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:10.515 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:10.516 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:15.521 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:15.522 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:15.523 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:19.018 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:19.019 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:23.669 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:39:57.742 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:23.678 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:39:57.752 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:24.021 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:24.022 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:24.023 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:24.051 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:39:58.125 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:27.334 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:27.335 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:32.337 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:32.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:32.338 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:35.723 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:35.724 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:40.725 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:40.726 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:40.727 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:41.856 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:39:09.093 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:41.861 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:39:09.098 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:42.197 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:39:09.434 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:28:44.107 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:44.108 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:49.110 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:49.112 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:49.112 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:28:52.306 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:28:52.306 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:28:57.308 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:57.309 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:28:57.310 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:01.024 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:01.025 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:06.026 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:06.028 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:06.029 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:09.459 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:09.460 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:14.462 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:14.462 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:14.463 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:15.321 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:40:49.394 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:15.332 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:40:49.405 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:15.388 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:40:49.461 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:29:17.543 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:17.544 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:22.546 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:22.547 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:22.547 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:25.588 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:25.589 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:30.590 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:30.591 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:30.592 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:33.709 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:33.710 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:38.711 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:38.713 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:38.714 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:41.990 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:41.991 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:46.992 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:46.993 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:46.994 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:50.446 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:50.447 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:29:55.448 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:55.449 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:29:55.450 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:29:58.862 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:29:58.863 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:03.864 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:03.866 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:03.866 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:07.185 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:07.186 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:12.188 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:12.189 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:12.190 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:15.496 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:15.497 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:19.643 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:40:46.880 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:30:19.645 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:40:46.882 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:30:19.707 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:40:46.944 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:30:20.499 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:20.500 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:20.500 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:23.765 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:23.766 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:28.767 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:28.768 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:28.769 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:31.974 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:31.975 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:36.979 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:36.980 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:36.981 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:40.134 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:40.134 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:45.136 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:45.138 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:45.139 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:48.708 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:48.709 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:30:53.711 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:53.712 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:30:53.712 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:30:56.950 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:30:56.951 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:01.952 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:01.953 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:01.954 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:05.266 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:05.266 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:10.268 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:10.269 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:10.270 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:13.870 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:13.871 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:18.872 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:18.873 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:18.874 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:22.202 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:22.203 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:27.205 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:27.206 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:27.207 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:30.394 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:30.394 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:35.395 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:35.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:35.397 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:38.914 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:38.914 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:43.916 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:43.917 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:43.918 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:47.140 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:47.141 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:31:52.143 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:52.144 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:31:52.144 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:31:55.451 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:31:55.452 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:00.453 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:00.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:00.455 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:04.240 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:04.241 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:09.241 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:09.243 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:09.244 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:12.720 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:12.721 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:17.722 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:17.723 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:17.724 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:21.801 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:21.802 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:26.804 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:26.805 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:26.805 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:30.414 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:30.416 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:35.417 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:35.418 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:35.418 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:38.953 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:38.954 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:43.955 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:43.956 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:43.956 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:47.314 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:47.315 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:32:52.317 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:52.318 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:32:52.319 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:32:55.880 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:32:55.881 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:00.882 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:00.883 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:00.883 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:04.432 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:04.433 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:09.434 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:09.435 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:09.435 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:12.988 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:17.990 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:17.992 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:17.993 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:21.504 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:21.505 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:26.510 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:26.511 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:26.511 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:30.439 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:30.440 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:35.441 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:35.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:35.443 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:39.079 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:39.080 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:44.082 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:44.083 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:44.083 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:47.430 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:47.432 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:33:52.435 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:52.436 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:33:52.437 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:33:55.860 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:33:55.862 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:00.865 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:00.866 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:00.866 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:04.657 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:04.658 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:09.660 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:09.661 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:09.662 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:12.790 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:12.791 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:17.793 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:17.794 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:17.795 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:21.527 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:21.528 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:26.529 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:26.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:26.531 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:30.070 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:30.071 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:35.072 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:35.073 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:35.074 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:38.471 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:38.472 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:43.480 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:43.482 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:43.483 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:46.896 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:46.897 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:34:51.899 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:51.899 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:34:51.900 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:34:55.290 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"init","num_shards":0,"instance_id":0},"markers":[]},"full_sync":{"total":0,"complete":0}} 2020-05-24T15:34:55.291 ERROR:rgw_multi.tests:len(log_status)=4 len(sync_status)=0 2020-05-24T15:35:00.293 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_sync_disable ... FAIL 2020-05-24T15:35:00.295 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-59 2020-05-24T15:35:01.131 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-60 2020-05-24T15:35:02.373 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-61 2020-05-24T15:35:14.529 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:14.530 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:14.531 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:18.046 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:46:52.119 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:18.053 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:46:52.122 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:18.329 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:46:52.402 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:19.653 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.12.5"}]} 2020-05-24T15:35:19.654 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-59 2020-05-24T15:35:19.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:19.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:19.656 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:23.173 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.12.5"}}}] 2020-05-24T15:35:23.174 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-59 2020-05-24T15:35:23.175 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:23.175 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:23.176 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:25.934 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.12.5"}]} 2020-05-24T15:35:25.934 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-59 2020-05-24T15:35:25.935 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:25.936 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:25.936 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:28.913 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.12.5"}}}] 2020-05-24T15:35:28.913 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-59 2020-05-24T15:35:28.914 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:28.915 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:28.915 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:31.906 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.12.5"}]} 2020-05-24T15:35:31.907 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-59 2020-05-24T15:35:31.907 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:31.908 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:31.908 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:35.430 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.12.5"}}}] 2020-05-24T15:35:35.431 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-59 2020-05-24T15:35:35.432 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:35.433 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:35.434 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:36.443 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:46:03.681 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:36.449 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:46:03.687 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:36.698 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:46:03.936 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:35:38.313 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.9.5"}]} 2020-05-24T15:35:38.314 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-59 2020-05-24T15:35:38.315 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:38.315 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:38.316 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:41.125 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.9.5"}}}] 2020-05-24T15:35:41.126 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-59 2020-05-24T15:35:41.127 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:41.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:41.129 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:43.756 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.9.5"}]} 2020-05-24T15:35:43.757 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-59 2020-05-24T15:35:43.758 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:43.758 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:43.759 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:46.505 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.9.5"}}}] 2020-05-24T15:35:46.506 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-59 2020-05-24T15:35:46.507 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:46.507 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:46.508 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:49.378 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.9.5"}]} 2020-05-24T15:35:49.379 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-59 2020-05-24T15:35:49.379 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:49.381 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:49.381 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:35:52.974 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:35:52.975 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:35:57.977 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:57.978 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:35:57.979 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:01.829 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:01.835 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:36:06.857 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:06.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:06.860 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:10.577 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:10.578 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:36:15.356 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:46:42.587 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:36:15.356 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:46:42.589 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:36:15.394 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:46:42.631 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:36:15.580 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:15.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:15.582 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:19.591 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:19.592 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:36:24.594 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:24.595 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:24.595 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:27.919 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:27.920 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:36:32.922 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:32.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:32.923 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:36.217 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:36.218 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:36:41.219 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:41.220 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:41.221 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:44.586 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:44.586 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:36:49.588 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:49.589 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:49.590 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:36:52.928 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:36:52.929 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:36:57.931 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:57.933 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:36:57.934 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:01.524 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:01.525 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:37:06.526 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:06.527 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:06.528 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:10.558 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:10.559 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:37:15.560 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:15.562 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:15.562 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:18.870 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:18.871 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:37:23.874 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:23.875 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:23.876 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:27.793 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:27.794 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:37:32.795 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:32.796 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:32.797 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:36.275 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:49:10.347 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:36.277 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:49:10.350 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:36.352 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:49:10.425 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:37:36.365 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:36.366 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:37:41.368 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:41.369 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:41.369 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:44.808 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:44.809 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:37:49.810 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:49.811 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:49.812 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:37:53.250 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:37:53.251 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:37:58.252 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:58.254 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:37:58.255 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:01.995 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:01.996 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:38:06.998 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:06.999 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:07.000 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:10.653 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:10.654 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:38:15.656 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:15.657 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:15.658 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:18.955 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:18.956 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:38:21.660 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:49:55.732 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:21.672 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:49:55.744 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:21.742 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:49:55.814 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:23.958 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:23.959 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:23.960 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:27.434 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:27.435 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:38:32.437 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:32.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:32.439 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:35.705 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:35.706 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:38:40.686 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:49:07.923 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:40.708 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:40.709 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:40.709 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:40.719 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:49:07.957 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:40.811 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:49:08.048 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:38:44.342 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:44.343 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:38:49.345 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:49.345 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:49.346 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:38:52.910 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:38:52.910 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:38:57.912 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:57.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:38:57.926 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:01.343 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:01.344 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:39:06.346 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:06.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:06.349 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:10.170 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:10.171 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:39:15.173 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:15.174 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:15.175 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:18.762 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:18.763 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:39:23.764 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:23.765 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:23.766 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:27.333 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:27.334 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:39:32.335 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:32.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:32.337 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:35.876 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:35.877 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:39:40.879 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:40.881 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:40.882 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:44.651 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:44.652 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:39:49.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:49.657 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:49.657 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:39:53.115 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:39:53.116 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:39:58.117 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:58.117 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:39:58.118 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:02.075 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:02.076 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:40:07.077 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:07.078 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:07.079 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:10.613 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:10.614 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:40:15.616 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:15.617 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:15.617 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:18.953 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:18.953 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:40:23.955 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:23.956 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:23.957 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:27.205 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:27.205 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:40:32.207 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:32.207 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:32.208 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:35.995 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:35.995 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:40:40.997 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:40.998 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:40.999 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:44.730 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:44.730 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:40:49.732 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:49.733 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:49.734 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:40:53.300 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:40:53.301 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:40:58.302 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:58.304 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:40:58.304 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:01.823 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:01.824 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:41:06.826 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:06.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:06.828 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:10.915 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:10.915 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:41:15.917 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:15.918 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:15.919 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:19.399 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:19.400 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:41:24.411 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:24.413 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:24.414 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:27.962 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:27.963 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:41:32.965 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:32.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:32.966 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:36.483 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:36.484 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:41:41.485 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:41.486 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:41.487 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:44.997 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:44.998 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:41:50.000 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:50.000 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:50.001 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:41:53.291 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:41:53.291 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:41:58.292 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:58.293 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:41:58.294 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:01.869 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:01.870 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:42:06.871 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:06.872 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:06.873 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:10.770 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:10.771 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:42:12.672 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:53:46.744 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:12.689 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:53:46.762 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:12.934 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:53:47.006 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:15.772 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:15.773 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:15.774 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:19.438 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:19.439 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:42:24.440 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:24.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:24.927 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:28.312 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:28.313 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:42:30.872 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:52:58.109 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:30.880 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:52:58.117 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:31.097 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:52:58.335 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:42:33.314 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:33.315 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:33.316 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:36.590 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:36.591 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:42:41.592 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:41.593 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:41.594 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:45.173 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:45.174 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:42:50.176 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:50.178 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:50.186 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:42:53.581 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:42:53.581 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:42:58.584 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:58.585 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:42:58.586 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:01.922 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:01.923 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:43:06.924 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:06.925 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:06.926 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:10.807 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:10.807 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:43:15.811 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:15.812 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:15.812 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:19.346 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:19.347 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:43:24.354 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:24.355 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:24.356 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:28.117 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:28.118 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:43:33.119 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:33.120 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:33.120 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:36.968 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:36.968 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:43:41.969 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:41.970 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:41.971 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:45.372 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:45.374 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:43:50.375 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:50.376 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:50.377 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:43:53.888 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:43:53.888 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:43:58.890 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:58.891 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:43:58.892 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:02.699 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:02.700 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:44:07.701 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:07.703 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:07.704 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:11.433 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:11.434 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:44:16.436 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:16.437 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-59', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:16.438 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-59 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:20.097 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:44:20.097 WARNING:rgw_multi.tests:bucket ajxnhd-59 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.9.5 target= 2020-05-24T15:44:25.100 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_sync_enable_right_after_disable ... FAIL 2020-05-24T15:44:25.102 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-62 2020-05-24T15:44:26.053 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-63 2020-05-24T15:44:28.611 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-64 2020-05-24T15:44:46.236 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:46.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:46.237 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:51.900 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=f35208d4-7215-4668-86dd-a246cde34636 2020-05-24T15:44:53.095 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590335070.844949_268.1","last_update":"2020-05-24 15:44:30.844949Z"},{"marker":"","last_update":"0.000000"}] 2020-05-24T15:44:53.096 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T15:44:53.096 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:53.097 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:53.097 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:55.631 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590335070.844949_268.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:44:30.844949Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:44:55.632 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T15:44:55.632 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T15:44:55.633 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:55.633 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:55.634 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:44:59.298 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590335070.844949_268.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:44:30.844949Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:44:59.299 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T15:44:59.299 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T15:44:59.300 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:59.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:44:59.301 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:03.581 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590335070.844949_268.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 15:44:30.844949Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T15:45:03.582 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T15:45:03.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:03.583 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:03.584 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:08.045 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.24.5"}]} 2020-05-24T15:45:08.045 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-62 2020-05-24T15:45:08.046 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:08.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:08.047 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:11.480 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.24.5"}}}] 2020-05-24T15:45:11.480 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-62 2020-05-24T15:45:11.481 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:11.481 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:11.482 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:14.760 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.24.5"}]} 2020-05-24T15:45:14.761 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-62 2020-05-24T15:45:14.761 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:14.762 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:14.762 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:17.655 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.24.5"}}}] 2020-05-24T15:45:17.656 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-62 2020-05-24T15:45:17.657 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:17.657 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:17.658 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:20.395 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.24.5"}]} 2020-05-24T15:45:20.396 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-62 2020-05-24T15:45:20.397 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:20.398 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:20.399 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:24.513 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.24.5"}}}] 2020-05-24T15:45:24.514 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-62 2020-05-24T15:45:24.515 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:24.515 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:24.516 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:27.562 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.24.5"}]} 2020-05-24T15:45:27.563 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-62 2020-05-24T15:45:27.563 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:27.564 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:27.564 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:30.809 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.24.5"}}}] 2020-05-24T15:45:30.810 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-62 2020-05-24T15:45:30.810 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:30.811 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:30.812 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:33.325 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.24.5"}]} 2020-05-24T15:45:33.326 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-62 2020-05-24T15:45:33.327 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:33.327 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:33.328 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:36.482 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"obj4","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.24.5"}}}] 2020-05-24T15:45:36.482 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-62 2020-05-24T15:45:36.483 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:36.484 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:36.484 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:39.509 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.24.5"}]} 2020-05-24T15:45:39.510 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-62 2020-05-24T15:45:39.511 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:39.511 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:39.512 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:43.031 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:45:43.032 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:45:48.033 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:48.035 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:48.035 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:51.926 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:45:51.927 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:45:56.929 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:56.930 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:45:56.930 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:45:57.444 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:57:31.516 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:45:57.446 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:57:31.519 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:45:57.628 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:01:57:31.700 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:46:00.319 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:00.320 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:46:05.321 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:05.322 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:05.322 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:08.638 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:08.639 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:46:13.641 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:13.642 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:13.643 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:17.317 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:17.318 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:46:22.319 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:22.320 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:22.321 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:25.672 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:25.681 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:46:30.682 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:30.683 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:30.684 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:34.412 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:34.413 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:46:39.414 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:39.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:39.415 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:43.096 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:43.097 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:46:48.098 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:48.099 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:48.100 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:46:51.564 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:46:51.565 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:46:56.566 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:56.567 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:46:56.568 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:00.467 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:00.467 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:47:01.947 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:57:29.185 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:47:01.949 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:57:29.187 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:47:02.081 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:57:29.318 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:47:05.469 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:05.470 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:05.471 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:09.055 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:09.056 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:47:14.057 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:14.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:14.059 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:17.705 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:17.706 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:47:22.707 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:22.708 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:22.709 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:26.318 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:26.319 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:47:31.321 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:31.322 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:31.323 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:35.029 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:35.030 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:47:40.031 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:40.033 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:40.034 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:43.607 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:43.608 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:47:48.609 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:48.610 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:48.611 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:47:52.460 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:47:52.461 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:47:57.462 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:57.463 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:47:57.464 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:01.417 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:01.418 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:48:06.420 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:06.422 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:06.423 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:09.933 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:09.934 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:48:14.935 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:14.936 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:14.937 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:18.769 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:18.770 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:48:23.771 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:23.773 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:23.774 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:27.676 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:27.677 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:48:32.679 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:32.680 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:32.680 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:36.688 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:36.689 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:48:41.690 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:41.692 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:41.692 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:45.033 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:45.034 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:48:50.036 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:50.037 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:50.038 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:48:53.617 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:48:53.618 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:48:58.620 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:58.621 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:48:58.622 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:02.335 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:02.336 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:49:06.665 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:00:40.737 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:06.695 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:00:40.768 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:06.915 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:00:40.988 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:07.337 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:07.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:07.338 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:11.115 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:11.116 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:49:16.118 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:16.119 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:16.119 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:19.783 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:19.784 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:49:24.785 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:24.786 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:24.787 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:25.206 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:59:52.443 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:25.219 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:59:52.456 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:25.368 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:01:59:52.605 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:49:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:28.966 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:49:33.968 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:33.969 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:33.970 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:37.745 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:37.746 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:49:42.748 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:42.749 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:42.750 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:46.218 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:46.219 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:49:51.220 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:51.221 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:49:51.221 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:49:55.067 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:49:55.068 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:50:00.072 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:00.073 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:00.074 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:03.494 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:03.495 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:50:08.497 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:08.499 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:08.499 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:12.609 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:12.610 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:50:17.611 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:17.612 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:17.613 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:21.060 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:21.061 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:50:26.062 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:26.063 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:26.064 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:29.462 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:29.463 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:50:34.464 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:34.465 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:34.466 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:38.468 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:38.469 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:50:43.470 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:43.472 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:43.473 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:47.227 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:47.228 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:50:52.229 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:52.230 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:50:52.230 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:50:56.027 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:50:56.028 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:51:01.029 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:01.031 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:01.032 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:04.688 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:04.688 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:51:09.690 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:09.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:09.692 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:13.219 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:13.220 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:51:18.221 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:18.223 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:18.223 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:21.832 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:21.833 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:51:26.834 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:26.835 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:26.835 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:30.537 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:30.538 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:51:35.540 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:35.541 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:35.542 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:39.694 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:39.695 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:51:44.698 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:44.699 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:44.700 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:48.494 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:48.494 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:51:53.496 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:53.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:51:53.499 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:51:57.144 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:51:57.145 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:52:02.147 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:02.148 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:02.148 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:05.792 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:05.793 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:52:10.794 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:10.795 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:10.796 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:14.544 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:14.545 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:52:19.547 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:19.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:19.549 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:23.238 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:23.239 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:52:28.242 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:28.244 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:28.245 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:32.098 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:32.099 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:52:37.101 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:37.102 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:37.102 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:40.763 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:40.900 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:52:45.901 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:45.902 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:45.903 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:49.643 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:49.643 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:52:54.645 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:54.646 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:52:54.647 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:52:58.689 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:52:58.690 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:53:03.694 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:03.695 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:03.696 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:07.536 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:07.537 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:53:12.538 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:12.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:12.540 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:16.197 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:16.197 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:53:21.198 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:21.199 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:21.200 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:24.635 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:24.636 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:53:29.638 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:29.640 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:29.641 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:33.458 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:33.459 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:53:38.461 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:38.462 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:38.463 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:41.977 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:41.978 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:53:46.979 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:46.980 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:46.981 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:50.594 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:50.595 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:53:55.596 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:55.597 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:53:55.598 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:53:59.032 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:53:59.033 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:54:04.034 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:04.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:04.036 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:07.736 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:54:07.736 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:54:12.738 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:12.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:12.740 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-62 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:16.716 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:54:16.717 WARNING:rgw_multi.tests:bucket ajxnhd-62 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.24.5 target= 2020-05-24T15:54:18.518 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:05:52.591 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:54:18.522 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:05:52.595 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:54:18.727 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:05:52.799 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:54:21.719 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_sync_disable_enable ... FAIL 2020-05-24T15:54:21.720 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-65 2020-05-24T15:54:22.647 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=ajxnhd-66 2020-05-24T15:54:23.850 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=ajxnhd-67 2020-05-24T15:54:49.035 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:49.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:49.037 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:54.747 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000018.20.5"}]} 2020-05-24T15:54:54.748 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-65 2020-05-24T15:54:54.749 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:54.749 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:54:54.750 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:54:57.444 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000017.19.4"}}}] 2020-05-24T15:54:57.445 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone2 behind zone test-zone1: shard=0 master=00000000018.20.5 target=00000000017.19.4 2020-05-24T15:55:02.446 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:02.447 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:02.448 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:06.049 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000018.20.5"}}}] 2020-05-24T15:55:06.050 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-65 2020-05-24T15:55:06.051 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:06.052 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:06.052 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:09.706 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000018.20.5"}]} 2020-05-24T15:55:09.707 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-65 2020-05-24T15:55:09.707 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:09.708 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:09.709 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:13.783 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000018.20.5"}}}] 2020-05-24T15:55:13.784 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-65 2020-05-24T15:55:13.785 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:13.786 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:13.786 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:16.846 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000018.20.5"}]} 2020-05-24T15:55:16.847 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-65 2020-05-24T15:55:16.848 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:16.848 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:16.849 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:20.903 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":"00000000018.20.5"}}}] 2020-05-24T15:55:20.903 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-65 2020-05-24T15:55:20.904 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:20.905 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:20.908 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:22.834 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:05:50.071 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:22.869 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:05:50.106 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:23.033 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:05:50.270 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:24.596 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.9.5"}]} 2020-05-24T15:55:24.597 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-65 2020-05-24T15:55:24.597 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:24.598 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:24.599 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:27.391 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"MULTIPART","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.9.5"}}}] 2020-05-24T15:55:27.391 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-65 2020-05-24T15:55:27.392 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:27.392 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:27.393 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:30.005 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.9.5"}]} 2020-05-24T15:55:30.005 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-65 2020-05-24T15:55:30.006 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:30.007 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:30.007 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:32.677 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"MULTIPART","instance":"","ns":""},"count":1},"inc_marker":{"position":"0#00000000002.9.5"}}}] 2020-05-24T15:55:32.678 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-65 2020-05-24T15:55:32.678 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:32.679 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:32.679 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:35.428 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000002.9.5"}]} 2020-05-24T15:55:35.429 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-65 2020-05-24T15:55:35.430 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:35.430 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:35.431 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:39.026 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:55:39.027 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:55:41.466 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:07:15.539 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:41.478 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:07:15.546 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:41.638 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:07:15.711 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T15:55:44.029 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:44.030 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:44.031 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:47.846 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:55:47.847 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:55:52.849 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:52.851 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:55:52.851 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:55:56.635 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:55:56.636 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:56:01.637 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:01.638 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:01.639 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:05.158 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:05.159 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:56:10.161 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:10.162 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:10.163 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:14.035 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:14.036 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:56:19.038 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:19.039 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:19.039 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:19.544 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:06:46.782 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:56:19.548 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:06:46.786 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:56:19.616 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:06:46.854 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T15:56:22.588 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:22.589 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:56:27.591 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:27.592 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:27.592 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:31.495 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:31.496 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:56:36.498 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:36.499 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:36.500 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:40.344 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:40.345 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:56:45.347 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:45.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:45.349 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:49.018 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:49.018 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:56:54.020 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:54.021 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:56:54.022 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:56:57.755 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:56:57.756 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:57:02.759 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:02.760 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:02.761 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:06.730 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:06.730 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:57:11.732 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:11.734 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:11.735 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:15.576 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:15.577 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:57:20.579 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:20.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:20.581 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:24.147 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:24.147 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:57:29.150 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:29.151 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:29.151 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:33.109 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:33.110 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:57:38.111 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:38.112 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:38.113 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:41.715 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:41.716 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:57:46.717 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:46.719 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:46.720 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:50.558 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:50.558 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:57:55.560 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:55.561 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:57:55.562 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:57:59.281 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:57:59.282 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:58:04.283 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:04.284 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:04.285 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:08.032 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:08.033 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:58:13.034 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:13.035 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:13.036 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:17.196 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:17.197 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:58:22.199 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:22.201 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:22.202 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:25.962 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:25.963 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:58:30.965 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:30.972 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:30.973 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:34.670 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:34.671 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:58:39.672 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:39.673 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:39.674 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:43.190 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:43.191 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:58:48.192 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:48.193 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:48.193 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:58:52.501 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:58:52.502 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:58:57.504 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:57.505 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:58:57.505 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:01.743 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:01.744 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:59:06.745 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:06.746 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:06.747 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:10.632 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:10.633 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:59:15.634 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:15.635 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:15.636 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:19.393 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:19.394 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:59:24.396 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:24.397 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:24.398 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:28.034 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:28.035 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:59:33.037 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:33.038 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:33.039 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:36.930 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:36.931 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:59:41.933 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:41.934 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:41.935 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:45.757 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:45.758 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:59:50.760 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:50.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:50.762 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T15:59:54.361 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T15:59:54.362 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T15:59:59.363 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:59.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T15:59:59.365 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:03.293 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:03.293 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:00:08.303 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:08.304 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:08.304 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:12.102 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:12.103 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:00:17.114 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:17.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:17.116 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:21.058 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:21.059 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:00:26.060 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:26.061 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:26.062 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:30.062 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:30.062 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:00:35.064 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:35.065 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:35.066 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:38.764 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:38.764 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:00:43.765 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:43.767 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:43.768 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:47.138 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:47.139 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:00:52.140 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:52.141 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:00:52.142 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:00:55.757 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:00:55.758 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:01:00.760 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:00.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:00.762 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:04.234 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:04.234 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:01:09.236 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:09.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:09.238 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:13.257 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:13.258 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:01:18.262 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:18.263 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:18.264 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:22.041 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:22.041 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:01:27.043 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:27.045 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:27.046 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:31.271 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:31.271 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:01:36.273 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:36.274 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:36.275 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:40.279 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:40.280 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:01:45.282 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:45.283 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:45.284 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:49.279 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:49.279 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:01:54.281 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:54.282 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:01:54.283 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:01:58.115 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:01:58.115 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:02:03.117 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:03.118 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:03.119 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:06.848 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:06.849 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:02:11.851 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:11.852 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:11.852 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:15.647 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:15.648 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:02:20.650 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:20.651 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:20.652 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:24.215 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:24.215 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:02:29.216 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:29.217 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:29.218 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:33.371 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:33.372 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:02:37.961 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:14:12.033 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:02:37.975 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:14:12.048 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:02:38.373 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:38.374 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:38.375 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:38.394 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:14:12.467 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:02:40.416 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:14:14.489 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:02:40.418 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:14:14.491 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:02:40.433 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:14:14.506 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:02:42.072 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:42.073 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:02:47.075 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:47.076 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:47.077 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:50.597 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:50.598 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:02:55.600 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:55.601 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:02:55.603 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:02:59.683 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:02:59.684 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:03:04.685 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:04.687 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:04.687 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:08.590 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:08.590 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:03:13.592 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:13.592 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:13.593 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:15.938 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:13:43.175 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:15.942 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:13:43.179 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:16.264 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:13:43.501 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:17.726 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:17.727 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:03:22.728 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:22.729 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:22.730 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:26.408 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:26.409 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:03:31.410 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:31.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:31.412 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:35.056 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:35.057 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:03:40.059 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:40.060 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:40.060 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:43.682 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:14:10.919 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:43.689 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:14:10.927 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:43.731 INFO:tasks.ceph.c2.mon.a.smithi160.stderr:==00:02:14:10.969 9733== Warning: unimplemented fcntl command: 1036 2020-05-24T16:03:44.049 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:44.049 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:03:49.051 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:49.052 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:49.053 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:03:52.840 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:03:52.840 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:03:57.842 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:57.843 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:03:57.843 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:01.634 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:04:01.635 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:04:06.636 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:06.638 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:06.638 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:10.281 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:04:10.282 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:04:15.283 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:15.284 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-65', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:15.285 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-65 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:19.140 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:04:19.141 WARNING:rgw_multi.tests:bucket ajxnhd-65 zone test-zone4 behind zone test-zone2: shard=0 master=00000000002.9.5 target= 2020-05-24T16:04:24.144 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_multipart_object_sync ... FAIL 2020-05-24T16:04:24.146 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-68 2020-05-24T16:04:25.980 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:25.981 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:25.982 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:29.428 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=f35208d4-7215-4668-86dd-a246cde34636 2020-05-24T16:04:30.053 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590336264.894762_288.1","last_update":"2020-05-24 16:04:24.894762Z"},{"marker":"","last_update":"0.000000"}] 2020-05-24T16:04:30.054 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T16:04:30.055 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:30.055 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:30.056 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:33.064 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590336264.894762_288.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:24.894762Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:04:33.065 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T16:04:33.066 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T16:04:33.066 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:33.070 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:33.071 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:36.854 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590336264.894762_288.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:24.894762Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:04:36.855 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T16:04:36.856 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T16:04:36.856 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:36.857 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:36.858 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:40.442 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590336264.894762_288.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:24.894762Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:04:40.443 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T16:04:40.444 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-68', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:40.444 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-68', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:40.445 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-68 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:44.470 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000004.6.5"}]} 2020-05-24T16:04:44.471 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-68 2020-05-24T16:04:44.472 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-68', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:44.472 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-68', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:44.473 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-68 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:47.381 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"testobj-sse-kms","instance":"","ns":""},"count":2},"inc_marker":{"position":"0#00000000004.6.5"}}}] 2020-05-24T16:04:47.382 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-68 2020-05-24T16:04:47.890 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_encrypted_object_sync ... ok 2020-05-24T16:04:47.891 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=ajxnhd-69 2020-05-24T16:04:51.697 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:51.698 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:51.699 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:04:57.171 INFO:teuthology.orchestra.run.smithi050.stderr:No --period given, using current period=f35208d4-7215-4668-86dd-a246cde34636 2020-05-24T16:04:58.322 INFO:teuthology.orchestra.run.smithi050.stdout:[{"marker":"","last_update":"0.000000"},{"marker":"","last_update":"0.000000"},{"marker":"1_1590336289.380365_292.1","last_update":"2020-05-24 16:04:49.380365Z"},{"marker":"","last_update":"0.000000"}] 2020-05-24T16:04:58.322 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2020-05-24T16:04:58.323 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:58.324 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:04:58.324 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:01.602 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590336289.380365_292.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:49.380365Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:05:01.604 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2020-05-24T16:05:01.605 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2020-05-24T16:05:01.606 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:01.607 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:01.608 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:05.499 INFO:teuthology.orchestra.run.smithi050.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590336289.380365_292.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:49.380365Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:05:05.500 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2020-05-24T16:05:05.500 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2020-05-24T16:05:05.501 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:05.501 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:05.502 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:10.246 INFO:teuthology.orchestra.run.smithi160.stdout:{"sync_status":{"info":{"status":"sync","num_shards":4,"period":"f35208d4-7215-4668-86dd-a246cde34636","realm_epoch":4},"markers":[{"key":0,"val":{"state":1,"marker":"1_1590328568.276896_39.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:56:08.276896Z","realm_epoch":2}},{"key":1,"val":{"state":1,"marker":"","next_step_marker":"","total_entries":1,"pos":0,"timestamp":"0.000000","realm_epoch":2}},{"key":2,"val":{"state":1,"marker":"1_1590336289.380365_292.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 16:04:49.380365Z","realm_epoch":4}},{"key":3,"val":{"state":1,"marker":"1_1590328767.216616_146.1","next_step_marker":"","total_entries":0,"pos":0,"timestamp":"2020-05-24 13:59:27.216616Z","realm_epoch":2}}]},"full_sync":{"total":1,"complete":1}} 2020-05-24T16:05:10.247 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2020-05-24T16:05:10.247 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:10.248 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:10.249 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:13.531 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.26.5"}]} 2020-05-24T16:05:13.531 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-69 2020-05-24T16:05:13.532 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:13.532 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:13.533 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:17.018 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.26.5"}}}] 2020-05-24T16:05:17.019 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone2 source_zone=test-zone1 bucket=ajxnhd-69 2020-05-24T16:05:17.019 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:17.020 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:17.021 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:19.941 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.26.5"}]} 2020-05-24T16:05:19.942 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-69 2020-05-24T16:05:19.942 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:19.943 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:19.943 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:22.699 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.26.5"}}}] 2020-05-24T16:05:22.700 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone1 bucket=ajxnhd-69 2020-05-24T16:05:22.701 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:22.701 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:22.702 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:25.547 INFO:teuthology.orchestra.run.smithi050.stdout:{"markers":[{"key":0,"val":"00000000008.26.5"}]} 2020-05-24T16:05:25.548 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-69 2020-05-24T16:05:25.548 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:25.549 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:25.550 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:29.235 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":4},"inc_marker":{"position":"00000000008.26.5"}}}] 2020-05-24T16:05:29.236 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone4 source_zone=test-zone1 bucket=ajxnhd-69 2020-05-24T16:05:29.236 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:29.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:29.238 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:31.826 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.22.5"}]} 2020-05-24T16:05:31.827 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-69 2020-05-24T16:05:31.828 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:31.829 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:31.832 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:34.737 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.22.5"}}}] 2020-05-24T16:05:34.737 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone1 source_zone=test-zone2 bucket=ajxnhd-69 2020-05-24T16:05:34.738 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:34.738 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:34.739 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:37.477 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.22.5"}]} 2020-05-24T16:05:37.478 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-69 2020-05-24T16:05:37.478 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:37.479 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:37.479 INFO:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync markers --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:40.214 INFO:teuthology.orchestra.run.smithi050.stdout:[{"key":0,"val":{"status":"incremental-sync","full_marker":{"position":{"name":"d","instance":"","ns":""},"count":4},"inc_marker":{"position":"0#00000000008.22.5"}}}] 2020-05-24T16:05:40.215 INFO:rgw_multi.tests:finished bucket checkpoint for target_zone=test-zone3 source_zone=test-zone2 bucket=ajxnhd-69 2020-05-24T16:05:40.216 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:40.216 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:40.217 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:42.816 INFO:teuthology.orchestra.run.smithi160.stdout:{"markers":[{"key":0,"val":"00000000008.22.5"}]} 2020-05-24T16:05:42.817 INFO:rgw_multi.tests:starting bucket checkpoint for target_zone=test-zone4 source_zone=test-zone2 bucket=ajxnhd-69 2020-05-24T16:05:42.818 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:42.818 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:42.819 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:46.301 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:05:46.302 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:05:51.304 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:51.305 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:05:51.306 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:05:55.090 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:05:55.090 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:06:00.098 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:00.099 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:00.100 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:03.891 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:03.892 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:06:08.894 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:08.895 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:08.896 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:12.371 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:12.371 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:06:17.373 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:17.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:17.376 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:21.127 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:21.128 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:06:26.130 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:26.131 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:26.132 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:29.738 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:29.739 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:06:34.740 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:34.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:34.742 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:38.625 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:38.626 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:06:43.628 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:43.629 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:43.630 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:47.314 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:47.315 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:06:52.316 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:52.318 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:06:52.318 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:06:56.185 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:06:56.187 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:07:01.188 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:01.189 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:01.190 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:05.080 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:05.081 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:07:10.082 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:10.083 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:10.084 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:13.864 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:07:18.866 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:18.867 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:18.868 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:22.833 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:22.834 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:07:27.835 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:27.836 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:27.838 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:31.843 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:31.844 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:07:36.846 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:36.847 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:36.847 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:40.407 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:40.408 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:07:45.409 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:45.410 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:45.411 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:49.508 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:49.509 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:07:54.511 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:54.512 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:07:54.513 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:07:58.326 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:07:58.327 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:08:03.330 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:03.332 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:03.333 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:07.465 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:07.466 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:08:12.468 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:12.469 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:12.470 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:16.353 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:16.354 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:08:21.356 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:21.357 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:21.358 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:24.771 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:24.771 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:08:29.773 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:29.774 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:29.775 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:33.693 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:33.694 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:08:38.695 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:38.696 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:38.697 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:42.549 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:42.550 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:08:47.552 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:47.555 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:47.556 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:08:51.711 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:08:51.712 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:08:56.714 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:56.715 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:08:56.716 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:00.765 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:00.766 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:09:05.772 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:05.773 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:05.775 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:09.172 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:09.173 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:09:14.181 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:14.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:14.184 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:17.975 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:17.976 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:09:22.977 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:22.979 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:22.980 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:26.776 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:26.776 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:09:31.778 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:31.779 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:31.779 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:34.408 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:21:08.478 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:09:34.411 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:21:08.484 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:09:34.765 INFO:tasks.ceph.c1.mon.a.smithi050.stderr:==00:02:21:08.838 9596== Warning: unimplemented fcntl command: 1036 2020-05-24T16:09:35.656 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:35.657 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:09:40.666 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:40.667 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:40.668 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:44.202 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:44.203 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:09:49.204 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:49.205 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:49.206 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:52.820 INFO:teuthology.orchestra.run.smithi160.stdout:[{"key":0,"val":{"status":"init","full_marker":{"position":{"name":"","instance":"","ns":""},"count":0},"inc_marker":{"position":""}}}] 2020-05-24T16:09:52.821 WARNING:rgw_multi.tests:bucket ajxnhd-69 zone test-zone4 behind zone test-zone2: shard=0 master=00000000008.22.5 target= 2020-05-24T16:09:57.823 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:57.824 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'markers', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--bucket', 'ajxnhd-69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2020-05-24T16:09:57.825 INFO:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync markers --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --bucket ajxnhd-69 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2020-05-24T16:09:57.899 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-05-24T16:09:57.903 DEBUG:teuthology.task.console_log:Killing console logger for smithi050 2020-05-24T16:09:57.906 DEBUG:teuthology.task.console_log:Killing console logger for smithi160 2020-05-24T16:09:57.907 DEBUG:teuthology.task.console_log:Killing console logger for smithi050 2020-05-24T16:09:57.908 DEBUG:teuthology.task.console_log:Killing console logger for smithi160 2020-05-24T16:09:57.908 DEBUG:teuthology.exit:Finished running handlers